Agentic RAG

The routing agent is the entry point. Its job is to look at a question and decide: what kind of information does this need, where does that information live, and should I search one source or several in sequence? For a simple factual question it might go directly to vector search. For a financial comparison it might query a structured database first for the raw numbers, then search documents for context. For a recent event it might skip the internal knowledge base entirely and go to web search. It also manages the iteration cycle — if a search returned empty or irrelevant results, the routing agent rewrites the query and tries a different approach rather than giving up.

A major limitation of simple RAG is assuming all knowledge lives in one place — a vector database full of embedded documents. Real organizations have knowledge spread across many different types of storage, each of which is the best tool for a different kind of question. Agentic RAG exposes each storage system as a tool that the routing agent can call. The agent picks the right tool for the query, or calls several in sequence if the question spans multiple sources.

The most important question in any RAG system is not "did I find something?" — it is "did I find the right thing?" A vector database will always return results; whether those results actually answer the question is a separate matter. Without a validation step, the system can confidently generate an answer that sounds plausible but is wrong, because the model filled in gaps from its training data rather than from the retrieved context. A validator agent reads the retrieved chunks and the draft answer together and asks: does the answer follow from what was actually retrieved? Is there anything in the sources that contradicts the answer? Are all the claims traceable to a specific retrieved passage, or did the model invent some of them?