Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails
Zhou Yu, Bin Bi and colleagues at Salesforce AI evolve an agent harness with a weak model, then show that fine-tuning that same weak model on a stronger expert's trajectories under the evolved harness makes it worse on all seven enterprise tasks, and give an on-policy correction pipeline that avoids the regression.
Ask this paper
The negative result is the contribution: Training the weaker model on the expert's complete trajectories under the evolved harness regresses performance by 4 to 30 points on all seven tasks across Qwen3-Coder and Gemma 4, even though the identical procedure helps under the unevolved harness.
Why imitation breaks the fit: The analysis attributes this to disrupted model-harness fit: the weak model adopts the expert's planning strategy without the competence to execute it, and then no longer matches a harness that was evolved around its own native planning style.
Harness evolution alone leaves a gap: Evolving the harness with the weak model helps, but a stronger expert then uses that same harness more effectively, which is what motivated adding expert supervision in the first place.
Rewrite one turn, not the trajectory: A meta-level MLE agent localizes the failing turn in the weak model's own rollout and asks the expert to rewrite only that turn, which preserves the model's planning style while still transferring the correction.
A recipe for economical co-evolution: Combining on-policy correction with harness evolution captures both gains, giving a compatibility-preserving procedure for domain-specific enterprise agents at a fraction of frontier-model cost.
Abstract
Agent harnesses (the system prompt, tool set, execution hooks, and context-management scaffolding around a model) are a critical determinant of agentic task success. Automated harness evolution can enable smaller models to perform well on domain-specific tasks at a fraction of frontier-model cost. Since both the harness and model weights shape behavior, we ask how harness evolution and lightweight fine-tuning should be combined. Across seven enterprise agent tasks, we first evolve a harness with the weaker model, then find that a stronger expert often uses it more effectively, suggesting expert supervision could close the remaining gap. However, training the weaker model on the expert's complete trajectories under the evolved harness backfires: performance regresses on all seven tasks by 4 to 30 points across Qwen3-Coder and Gemma 4, even though the same procedure helps under the unevolved harness. Our analysis shows that imitation transfers knowledge and increases scaffold usage, but disrupts model-harness fit: the weaker model adopts the expert's planning strategy without the competence to execute it and no longer matches the harness evolved around its native planning style. We therefore develop an on-policy expert-correction pipeline, automated by a meta-level MLE agent, that localizes the failing turn in the weaker model's own rollout and asks the expert to rewrite only that turn. This preserves the model's planning style and combines the gains of harness evolution and model adaptation. Our results identify and resolve a source of contention between harness and weight updates, yielding a compatibility-preserving recipe for economical co-evolution on domain-specific enterprise tasks.