HIRAG
Free while signed in. Answers cite the passages they came from.
HIRAG is a new instruction fine-tuning method that enhances the capabilities of RAG models by teaching them to think before answering. Key ideas:
Three Hierarchical Abilities – The authors propose that RAG models should possess three progressively hierarchical abilities: Filtering (selecting relevant information), Combination (combining information from multiple sources), and RAG-specific reasoning (making inferences from the provided documents).
Progressive Chain-of-Thought – HIRAG employs a "think before answering" strategy that uses a multi-level, progressive CoT to enhance the model's open-book examination capabilities. This allows the model to learn from easier to more complex tasks, significantly improving its performance in RAG scenarios.
Significant Performance Gains – Experiments show that the HIRAG training strategy significantly improves the model's performance on a variety of RAG datasets, including RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA.
Robust and Generalizable – The method is shown to be robust, with experiments on Chinese datasets confirming its effectiveness. Ablation studies also demonstrate that the training tasks for the three capabilities contribute to the performance of HIRAG.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack