Read

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis et al.

Artificial Intelligence Computer Science

This paper by Lewis et al. from Facebook AI Research introduces Retrieval-Augmented Generation (RAG), a general-purpose framework that combines a pre-trained parametric seq2seq model with a non-parametric dense vector index of Wikipedia, trained end-to-end. RAG enables language models to ground generation in retrieved documents, yielding state-of-the-art results on open-domain question answering and producing more factual and specific text than purely parametric baselines.

Read the paper

Key takeaways

RAG combines a Dense Passage Retriever that encodes queries and documents as BERT-based embeddings with a BART-large generator, treating retrieved documents as latent variables and marginalizing over them during training so that both components are jointly optimized without direct supervision on which documents to retrieve.
Two model variants are proposed: RAG-Sequence, which uses one retrieved document to generate the full output sequence, and RAG-Token, which can draw on a different document for each generated token, with the token-level variant performing better on tasks requiring synthesis across multiple sources.
RAG achieves state-of-the-art exact match scores on three open-domain QA datasets, including Natural Questions and WebQuestions, outperforming both parametric-only closed-book models and specialized retrieve-and-extract architectures that rely on retrieval supervision.
On the Jeopardy question generation task, human evaluators rated RAG generations as more factual than BART in 42.7% of cases versus 7.1% for BART, and RAG generations were judged more specific in 37.4% of cases versus 16.8% for BART.
For FEVER fact verification, RAG achieves accuracy within 4.3% of state-of-the-art pipeline models that use strong retrieval supervision and domain-specific architectures, despite RAG requiring no retrieval supervision.
The non-parametric memory in RAG can be updated at test time by swapping in a new document index without retraining, demonstrating that world knowledge can be refreshed independently of the generator parameters.

Reflections

This is the founding statement of a pattern almost everything I build now leans on, so I read it for the idea that survived rather than the specific retriever-and-generator machinery, which has long since been replaced. The durable insight is decoupling knowledge from the weights. Treat the documents as an external memory the model reads at inference time, and you can refresh what the system knows by swapping the index, with no retraining at all. That single property is why a system can stay current without being rebuilt. It’s also a useful corrective to the instinct to cram everything into the model itself. Some knowledge belongs in the parameters and some belongs in a store you can edit on Tuesday. Knowing which is which is most of the architecture. The model should reason; the index should remember.