DUET: Co-Evolving Solver and Grader Agents
Fengyu Gao (University of Virginia, Microsoft intern), Sourav Pal, Austin Z. Henley, Arjun Radhakrishna and Gustavo Soares at Microsoft introduce DUET, which optimizes a solver agent and the grader agent that scores it in alternating rounds, so the evaluator changes as the agent's failure modes change.
Ask this paper
Method. Each round selects a training batch that balances exploitation and exploration, runs the solver, has the grader score and critique the trajectories, and then a tool-using update module revises either the grader or the solver while the other stays fixed. Updates are bounded in size.
Solver results. On Finance Agent Benchmark DUET reaches a 96.5% rubric score against 92.8% unoptimized, 94.0% for SkillOpt and 94.4% for GEPA. It also beats SkillOpt and GEPA when those two optimize against the benchmark's own reference evaluator (95.7% and 95.9%).
Grader results. Grader pairwise agreement rises from 67.9% to 75.0% on Finance Agent Benchmark and from 68.4% to 78.5% on GDPval. Optimizing the grader alone with a fixed solver moves agreement only from 67.9% to 68.8%, so the two have to be trained together.
Skills too. Optimizing prompts and skills jointly reaches 91.9% success on tau-Bench against 83.3% for SkillOpt and 84.4% for GEPA. Starting from an adversarial solver prompt, DUET removes the harmful instructions in its first update.
Abstract
Agentic workflows are increasingly used across domains such as technology, finance, and enterprise operations. As these agents become more widely deployed, continually improving them becomes increasingly important. This raises an immediate challenge: How should the agent evolve? This evolution requires effective evaluation that can assess outcomes and provide useful feedback for optimization. As the agent evolves, its behaviors and failure modes may also change, making a fixed evaluator increasingly inadequate. Another fundamental question: How should we evaluate an evolving agent? These two challenges are inherently coupled; changes in agent behavior can expose limitations of the current evaluator, while a stronger evaluator provides more informative feedback for improving the agent. Motivated by this interaction, we introduce DUET, a framework that jointly optimizes a solver agent and a grader agent to improve both. DUET iteratively selects training tasks, executes them with the solver, evaluates the resulting outcomes with the grader, and uses a tool-using update module to revise the solver and the grader, alternating between the two across rounds. By updating the grader within the optimization loop, DUET turns evaluation from a fixed source of feedback into a first-class optimization objective that adapts alongside the solver. Experiments across four agent benchmarks show that DUET improves both solver and grader performance and consistently outperforms baselines that optimize the solver with a fixed grader.