🚀NEW LABGetting Started with Claude AgentsStart lab
Reinforcement Learning · Retrieval · Agents

DR Tulu

Figure 1
DR Tulu
The curator’s take

DR Tulu-8B is the first open model directly trained for long-form deep research using Reinforcement Learning with Evolving Rubrics (RLER). Unlike existing models trained on short-form QA tasks, DR Tulu learns to produce comprehensive, well-attributed research reports by training with rubrics that co-evolve with the model and ground on real-world searched knowledge. - **RLER training innovation:** The method generates new rubrics at each training step by contrasting multiple model rollouts and incorporating newly explored information from search results. This creates on-policy feedback that adapts as the model discovers new evidence, addressing the challenge that static rubrics cannot capture all quality dimensions for open-ended research tasks. - **Outperforms all open deep research models:** DR Tulu-8B beats existing 8-32B open models by 8-42 percentage points across four benchmarks (AstaBench-ScholarQA, DeepResearchBench, ResearchQA, HealthBench). It matches or exceeds proprietary systems like OpenAI Deep Research and Perplexity Deep Research while being significantly cheaper (USD 0.00008 vs USD 1.80 per query). - **Adaptive tool selection and search:** The model learns to choose appropriate search tools based on task type - using paper search 90% of the time on scientific questions (ResearchQA) but relying on web search 55% of the time for general-domain topics (DeepResearchBench), instead of using a single hard-coded tool. - **Full open release with MCP infrastructure:** Releases all training data, code, and models, plus a new MCP-based agent library (dr-agent-lib) with asynchronous tool calling support that makes it practical to train and evaluate deep research models at scale.

Ask this paper

Key points
01

RLER training innovation: The method generates new rubrics at each training step by contrasting multiple model rollouts and incorporating newly explored information from search results. This creates on-policy feedback that adapts as the model discovers new evidence, addressing the challenge that static rubrics cannot capture all quality dimensions for open-ended research tasks.

02

Outperforms all open deep research models: DR Tulu-8B beats existing 8-32B open models by 8-42 percentage points across four benchmarks (AstaBench-ScholarQA, DeepResearchBench, ResearchQA, HealthBench). It matches or exceeds proprietary systems like OpenAI Deep Research and Perplexity Deep Research while being significantly cheaper (USD 0.00008 vs USD 1.80 per query).

03

Adaptive tool selection and search: The model learns to choose appropriate search tools based on task type - using paper search 90% of the time on scientific questions (ResearchQA) but relying on web search 55% of the time for general-domain topics (DeepResearchBench), instead of using a single hard-coded tool.

04

Full open release with MCP infrastructure: Releases all training data, code, and models, plus a new MCP-based agent library (dr-agent-lib) with asynchronous tool calling support that makes it practical to train and evaluate deep research models at scale.

Every Monday
Get next week’s papers.
Subscribe on Substack