The New AI Stack

Happy Monday!

Last week, Dropbox Dash introduced major enhancements to its AI-powered search capabilities, combining traditional keyword search with semantic understanding to help users find information across fragmented work tools. But beneath this product update lies a fascinating story about how advanced information retrieval is becoming the backbone of truly useful AI systems.

The Building Blocks: RAG, IR, and the Search Revolution

Before we jump in, let's clarify some key terms: Information Retrieval (IR) is the science of finding relevant documents from a collection – it's what powers every search engine. Retrieval-Augmented Generation (RAG) is a technique where AI models first retrieve information from external sources before generating a response. This essentially gives AI the ability to "look things up" rather than relying solely on what it was trained on. Lexical search uses exact keyword matching (like traditional Google), while semantic search understands meaning regardless of specific words used (like when Google shows results that don't contain your exact query but are still relevant).

These technologies are now converging into sophisticated systems that are revolutionizing how AI interacts with information.

The Meta Trend: From Memorization to Retrieval

The focus of AI development has been creating larger models that memorized more information during training. But a noticeable shift is underway. Today's most advanced AI systems are evolving to use external knowledge on demand, rather than trying to memorize everything. This has broad implications for real-world applications of AI tools in established workflows.

Instead of just being limited to what they "learned in school" (their training data), modern AI systems can now:

1. Identify when they need additional information

2. Formulate effective search queries to find that information

3. Extract relevant details from search results

4. Synthesize this information with their existing knowledge

The implication is AI that's both more accurate and more current, able to draw on real-time information rather than potentially outdated training data. We are seeing pieces of this show up in all of the main offerings; deep research tools embed links and sources in all generated answers, and both Claude and ChatGPT have improved the ability to trace where and how responses are generated.

Pattern Recognition: Signs of the Search Revolution

Four patterns highlight how AI agents are leveraging advanced search to improve perform and capability:

1. RAG Becoming Standard Practice: Anthropic recently published detailed guidelines for building effective AI agents, highlighting RAG as a core component. They specifically note that "the retriever choice is critical" since it "sets the bounds of what your LLM can know at inference time." This recognition that retrieval quality directly impacts AI capability is driving major investment in search technology.

   Instead of viewing search as a separate function, companies are integrating it deeply into their AI architecture. The line between "search engine" and "AI assistant" is blurring.

2. Hybrid Search Approaches Dominating: Pure keyword search misses semantic connections; pure neural search can miss exact matches. The solution? Companies are increasingly adopting hybrid approaches that combine both techniques.

   Dropbox Dash, for example, uses a traditional keyword index to find candidate documents quickly, then applies neural embedding models to rank and filter them for relevance before sending the best matches to the LLM. This "best of both worlds" approach is becoming the industry standard because it delivers both precision and understanding.

3. Vector Databases Exploding in Adoption: The infrastructure powering semantic search, specialized vector databases, has seen astronomical growth. Companies like Pinecone, Weaviate, and Milvus have raised hundreds of millions in funding to build systems optimized for storing and searching through embeddings (when words essentially get translated into numbers for LLM’s to analyze and generate responses).

   Even traditional database players are getting in on the action. Elasticsearch added native vector search capabilities, and MongoDB launched Atlas Vector Search. This convergence signals that vector search is becoming as fundamental as traditional indexing.

4. Autonomy Through Better Search: As AI agents become more complex, advanced search serves as their primary tool for gathering information. Microsoft's ongoing development of Copilot showcases how AI that can effectively search through emails, documents, and meeting transcripts can perform complex tasks like summarizing project status or drafting reports.

   An AI assistant is ultimately only as good as the information it can access. Robust retrieval systems are what allow these assistants to deliver accurate, contextually relevant responses.

The Contrarian Take: Search Quality, Not Model Size

Growth in AI has historically centered on model scale, pushing for the “bigger is better” approach. But this misses a crucial point: even the largest models can only reason with the information available to them.

The real change happening now is the recognition that improving retrieval quality often yields more significant performance gains than simply scaling up model parameters. A smaller model with excellent retrieval capabilities can outperform a much larger model limited to its training data, especially for knowledge-intensive tasks.

This distinction matters because it shifts our focus from the purely technical challenge of building larger models to the more nuanced challenge of building better information retrieval systems. It's not just about how smart your AI is, but how effectively it can access the information it needs.

Practical Implications

For Investors:

• Look beyond model providers to the emerging ecosystem of retrieval technologies

• Vector databases and hybrid search platforms may be the "picks and shovels" of the AI revolution

• Companies with proprietary data that can be effectively indexed and searched could have significant advantages

For Enterprises:

• Consider how your organization's knowledge could be made more accessible to AI through better indexing

• Audit your existing search infrastructure; it may need significant upgrades to support modern AI applications

• Focus on the quality of information retrieval before investing heavily in custom model development

For Developers:

• Explore frameworks like LangChain and LlamaIndex that simplify implementing RAG pipelines

• Experiment with hybrid search approaches, don't just rely solely on embeddings or keywords

• Consider the entire retrieval pipeline: chunking strategies, reranking, and result fusion all impact final quality

The unification of search and AI represents an evolution in how we interact with information. Rather than separate tools, we're moving toward integrated systems that can seamlessly retrieve, reason, and respond.

As these technologies mature, we can expect AI assistants that feel remarkably more capable, being able to cite sources, draw from current information, and provide contextually nuanced responses. The days of AI hallucinating answers may be replaced by systems that know when to search for information and when to rely on built-in knowledge.

The companies that master this integration will likely lead the next wave of innovation.

In motion,
Justin Wright

1. China's Xi calls for self sufficiency in AI development amid U.S. rivalry (Reuters)

2. The Urgency of Interpretability (Dario Amodei)

3. How Discord Indexes Trillions of Messages (Discord)

4. Expanding on what we missed with sycophancy (OpenAI)

5. AI doctor learns to ‘see’ medical images (AI News)