AgentBaton: Efficient Model Collaboration in Agentic Tasks
Junlong Li (HKUST) and Jiarui Yao (UIUC), working as NVIDIA interns with Jindong Jiang, Yonggan Fu, Pavlo Molchanov and colleagues, study handing off a long-horizon agent task between a capable and a cheap model mid-run, and build AgentBaton, a detector that picks the handoff point.
Ask this paper
Direction matters. Handing off from a cheap model to a capable one generally underperforms capable-only runs and saves little or nothing. Handing off from capable to cheap at the right stage keeps or improves accuracy at much lower cost.
One handoff is enough. A single well-timed capable-to-efficient handoff performs well, so the problem reduces to choosing when to switch.
AgentBaton. A lightweight detector learns the handoff point from recurring task-progress patterns. It runs at a 338 ms median latency on one H100 for 2K to 121K token inputs.
Results. Across 11 model pairs and two benchmarks it cuts API cost by 65.9% on average while largely keeping accuracy, with gains of up to 8.8 points over capable-only runs, and extends the accuracy-cost Pareto frontier across reasoning-effort settings.
Abstract
Solving long-horizon agentic tasks requires language agents to interact with their environments over multiple steps, and reliable performance often comes from the expensive capable models available through cloud providers. Meanwhile, alternative efficient models with inferior performance but a much lower cost are emerging, including lower-tier cloud models and smaller open-weights models deployable on local hardware. This raises a natural question that remains less explored than traditional query-level routing: can capable and efficient models collaborate within a single long-horizon agentic task to substantially reduce cost while preserving task accuracy? In this paper, we systematically investigate key design choices for such collaboration through model handoffs, where one model takes over from another, focusing on handoff direction and context transfer. We find that efficient-to-capable handoffs generally underperform capable-only execution with limited or even no cost reduction, whereas capable-to-efficient handoffs at suitable stages of task progress largely preserve or even improve accuracy while substantially reducing cost. These findings suggest that a one-time, simple, well-timed handoff strategy is good enough for cost-effective agent collaboration, motivating AgentBaton, a lightweight detector that learns when to hand off from recurring patterns in task progress. Across 11 model pairs and two representative benchmarks, AgentBaton reduces API costs by 65.9% on average while largely preserving task accuracy, with accuracy gains of up to 8.8 percentage points over capable-only execution. Experiments across various reasoning-effort settings further show that AgentBaton extends the empirical accuracy-cost Pareto frontier.