Self-RAG
Free while signed in. Answers cite the passages they came from.
Self-RAG trains an LM to adaptively retrieve, generate, and self-critique using special reflection tokens.
Reflection tokens: Introduces special tokens that control retrieval decisions, passage relevance judgments, and self-evaluation of generations.
Adaptive retrieval: The model decides on-the-fly whether to retrieve, rather than always retrieving on every query - saving compute on knowledge-light queries.
Self-reflection: Critiques its own generations against retrieved passages, enabling controllable trade-offs between response quality and factuality at inference.
Significant gains: Outperforms state-of-the-art LLMs and strong RAG baselines on open-domain QA, reasoning, and fact verification.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack