SFR-DeepResearch
Free while signed in. Answers cite the passages they came from.
The paper introduces SFR-DeepResearch, a simple reinforcement-learning recipe that turns reasoning-optimized LLMs into autonomous single-agent researchers. The agent uses only three tools (search, static page browse, Python), manages its own context, and is trained end-to-end on synthetic short-form and long-form tasks with a length-normalized REINFORCE objective. Results show strong gains on FRAMES, GAIA, and Humanity’s Last Exam.
Agent design and scaffolding: Reformulates multi-turn tool use into a single, growing contextual question for QwQ and Qwen models, omitting earlier long CoTs to keep prompts stable. Adds a clean_memory tool to self-compress context when nearing limits.
Minimal toolset, fault tolerance: Tools are restricted to a bare search API, a static Markdown page scraper with no hyperlink clicking, and a stateless local Python interpreter, which makes training challenging enough to learn a strategy. Parsing and syntax errors trigger repair or retry routines to keep rollouts on track.
RL recipe: Uses synthetic, harder-than-Hotpot multi-hop QA plus report-writing tasks. Optimizes a group REINFORCE objective with temporal advantage normalization that divides by trajectory length, plus trajectory filtering and reuse of partial rollouts. Localized, cached tooling and a contamination blocklist stabilize training and evaluation.
Results: The best model, SFR-DR-20B, reaches 82.8 on FRAMES, 66.0 on GAIA (text-only), and 28.7 on HLE full text-only, outperforming comparable open agents and rivaling stronger proprietary systems under a contamination blocklist.
Ablations and behavior: The single-turn scaffolding beats default multi-turn templates for Qwen and QwQ, with large FRAMES gains. Length normalization curbs runaway tool calls that hurt reward and accuracy. Tool-use and token-length analysis shows gpt-oss-20B calls tools more yet writes much shorter per-step CoTs, indicating better token efficiency than Qwen-family models.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack