LM-Guided Chain-of-Thought
Free while signed in. Answers cite the passages they came from.
This paper offloads rationale generation to a small, trained LM while keeping a frozen large LM as the answer predictor. The small model is optimized with knowledge distillation and reinforcement learning so it produces rationales that steer the large model more effectively.
Split responsibilities: A small (<1B) LM writes the chain-of-thought rationale; a frozen large (>10B) LM reads the rationale and produces the final answer. Only the small LM is trained, cutting cost sharply.
Two-stage training: First, distill rationales from the large LM into the small one (knowledge distillation). Then fine-tune the small LM with RL using rationale-oriented and task-oriented reward signals.
Multi-hop QA gains: Evaluated on HotpotQA and 2WikiMultiHopQA, LM-guided CoT outperforms standard prompting and vanilla CoT prompting on answer-prediction accuracy; self-consistency decoding compounds the gains.
Cost-aware reasoning: The recipe is a pragmatic template for teams that cannot fine-tune frontier models - train a tiny rationale generator instead and leave the big model frozen as an API.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack