Retrieval-Augmented Generation

A language model on its own answers from what it absorbed during training — which is frozen at some past date, contains nothing about your organization, and cannot be cited. Retrieval-augmented generation fixes all three problems with the same move: before answering, search a document collection for passages relevant to the question, paste those into the prompt, and ask the model to answer from them.

The pipeline is search, then generate. Documents are split into chunks, converted into embeddings, and stored in a vector database. A question is embedded the same way, the closest chunks are retrieved, and the model answers using that material — usually with citations back to the sources.

RAG is how most real business AI gets built. It is what lets a model answer questions about your contracts, your policies, or last week's numbers without retraining anything, and it is dramatically cheaper and faster to update than fine-tuning — new documents are available the moment they are indexed.

It also makes answers checkable. A grounded answer comes with sources a human can verify, which converts an unauditable assertion into something you can actually govern.

When RAG disappoints, the failure is nearly always retrieval rather than generation: the right passage was never fetched, so the model never had a chance. Debug by looking at what was retrieved before blaming the model or reaching for a bigger one.