The Power of Noise: Redefining Retrieval in RAG
Free while signed in. Answers cite the passages they came from.
A study stress-testing the retriever component of RAG systems with surprising results about what actually helps generation.
Position matters: Relevant documents must be placed near the query for the LLM to attend to them - bury them and the model effectively ignores the evidence.
Related ≠ helpful: Documents that are topically related but not directly relevant can actively hurt RAG accuracy, counter to the common "retrieve broadly" heuristic.
Noise can help: Adding seemingly irrelevant or noisy passages in the right positions can boost accuracy, suggesting retrieval acts partly as a distractor regularizer.
Design implication: Retriever design should optimize for positional placement and document distinctiveness, not just topical similarity to the query.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack