This blog post uses the metaphor of a degraded library, where librarians push sales and books are filled with ads, to illustrate the declining quality of modern internet search engines plagued by SEO spam and affiliate links driven by financial incentives. The author, a former tech support professional, notes a significant shift from easily discovering unknown content to struggling even to find known information. While large language models (LLMs) like ChatGPT currently offer a seemingly cleaner, intent-driven alternative for "discovery search" without overt commercialization, the author raises a critical concern: the immense capital invested in AI development necessitates future monetization. The piece warns that these LLM providers, unlike public libraries, are businesses that will eventually seek massive returns, potentially leading to similar manipulative or exploitative monetization strategies, mirroring the very problems AI is currently solving in search.
Exploring the Internet's "Library" and the Future of AI-Driven Search
The Degraded Library Metaphor: The author introduces the metaphor of a vast, once-ideal library now overrun by pushy librarians selling irrelevant items (search engines/ads) and books filled with ads instead of content (SEO-optimized webpages).
Declining Search Quality: Drawing on personal anecdata from 2008-2012 and recent research, the author argues that search engine quality has significantly deteriorated, making it difficult to find relevant content even when its existence is known.
The SEO vs. Search Engine "Cat-and-Mouse Game": Research shows that search engines intervene, but SEO spam, especially pages with affiliate links (30-40% in results vs. 2.35% on the open internet), remains a pervasive problem due to strong financial incentives.
For-Profit vs. Public Service Librarians: The author questions the model of for-profit search engines with proprietary indexing, suggesting a public service model for search might be better, but acknowledges that human ingenuity in gaming systems could still pose a challenge.
Search for Known Content (Retrieval): While search engines can still find specific, known content (e.g., PostgreSQL commands), they often bury official documentation under ads, videos, or corporate blogs.
Challenges in Discovering Unknown Content: Formulating queries for true discovery is harder, and these searches are more susceptible to ads and SEO spam, making it difficult to explore broad topics (e.g., "cloud database").
AI for Discovery: LLMs like Claude and ChatGPT are currently more effective for "discovery searches" due to their ability to understand intent, engage in conversational queries, and their current lack of overt advertising, making fact-checking their mistakes less time-consuming than filtering SEO spam.
The Future of AI Monetization (The Puzzle): The author poses the critical question: Will LLMs also fall prey to monetization, weaponizing their conversational abilities for subtle manipulation (e.g., weaving product recommendations, creating artificial urgency, exploiting conversation flow, information shading) to generate revenue?
Massive AI Investment: Billions of dollars (e.g., OpenAI's projected $44 billion in losses until 2029, American Stargate's $500 billion plans) have been invested in the current wave of AI.
Investors Expect Returns: This immense capital demands significant financial returns, raising concerns about how LLM service providers will become profitable without resorting to "terrifying and dangerous" monetization schemes that could harm users and the broader industry built on AI services.
Ethical Implications: The author emphasizes that LLM providers are businesses, not public services, and "capitalism does not have built-in morals," urging consideration of the societal externalities that might arise from their profit-seeking endeavors.