The Power of Noise: Redefining Retrieval in RAG
A study stress-testing the retriever component of RAG systems with surprising results about what actually helps generation.
Ask this paper
Position matters: Relevant documents must be placed near the query for the LLM to attend to them - bury them and the model effectively ignores the evidence.
Related ≠ helpful: Documents that are topically related but not directly relevant can actively hurt RAG accuracy, counter to the common "retrieve broadly" heuristic.
Noise can help: Adding seemingly irrelevant or noisy passages in the right positions can boost accuracy, suggesting retrieval acts partly as a distractor regularizer.
Design implication: Retriever design should optimize for positional placement and document distinctiveness, not just topical similarity to the query.