Retriever-Free Retrieval: Navigating Documents with Decision Models
Chao Wan, Paul Jünger, Beixi Du, Bodong Liu, Navaneetha Krishnan and Kilian Weinberger (Cornell University, ASAPP) propose DecisionRAG, which drops the first-stage retriever and instead uses a decision model to navigate a corpus hierarchy from topics to documents to passages.
Ask this paper
Method. A decision model returns probabilities over answer options for many questions in one forward pass, without generating tokens. At each level of the corpus hierarchy it judges all candidates in a shared context and keeps the top branches, so pruned documents are never expanded.
Shared context. Scoring candidates together makes their relevance estimates comparable. It improves R@1 on MultiDoc2Dial by 7.7 points and Hit@1 on Natural Questions by 6.5 points over judging each candidate separately.
Results. Without task-specific training or a first-stage retriever, it beats Qwen3-Embedding-8B dense retrieval with an 8B reranker by 13.8 R@1 points on MultiDoc2Dial and 17.4 Hit@1 points on Natural Questions over 2.68 million Wikipedia paragraphs.
Cost and speed. Retrieval takes 0.7 to 1.8 seconds and 0.2 to 2.3 cents per query across four benchmarks. An LLM that generates navigation decisions token by token reaches similar accuracy at 34 to 145x the latency.
Trained judge. A Qwen3-8B decision model trained on MultiDoc2Dial data reaches 60.7 R@1, 5.7 points above the strongest published task-trained retriever.
Abstract
While language models have become dramatically more capable, retrieval still largely relies on lexical matching or similarity in fixed embedding spaces. This creates a capability asymmetry: what the retriever fails to find, even the strongest generator cannot recover. We propose DecisionRAG, which brings the capabilities of pretrained language models to retrieval using decision models that jointly evaluate candidate branches in a single forward pass, without generating any tokens, and show that judging candidates in a shared context makes their relevance estimates comparable. Like a person browsing a library, DecisionRAG navigates the corpus's inherent structure from broad topics down to individual passages, deciding at each level which branches are worth exploring. Without task-specific training or a first-stage retriever, DecisionRAG outperforms dense retrieval with an 8B reranker by 13.8 R@1 points on MultiDoc2Dial and 17.4 Hit@1 points on Natural Questions. Across four benchmarks, including a corpus of 2.68 million Wikipedia paragraphs, retrieval takes just 0.7-1.8 seconds per query.