Skip to content

Glossary

Retrieval-augmented generation (RAG)

Answering from documents handed to the model at question time, found by searching your own data, instead of from what it memorised during training.

Three steps: turn the question into a search, put the passages that come back into the prompt, and let the model answer out of them. The model learns nothing — the knowledge stays in your documents, and correcting it means editing a document, not retraining anything.

This is the usual answer to stale knowledge and to hallucination, but only the usual one, not an automatic one. RAG fails at the search step far more often than at the writing step: hand it the wrong passage and it will answer confidently from the wrong passage. Measuring retrieval separately from the answer is what separates a system that works from a demo that once did.

For knowledge that is narrow and does not move, fine-tuning is the alternative. The rough division: facts that change go into retrieval, form and tone go into training.