HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution
Wen Jiang and colleagues name three failure modes that break self-evolving agents and address all three with HarnessEvolve, which learns from reference trajectories generated by replaying tasks with the ground-truth answer in hand.
Ask this paper
Three named problems. Credit assignment failure from terminal-only feedback, shortcut learning where agents memorize task-specific patterns, and catastrophic forgetting where unguarded updates erase earlier competence.
Reference trajectories as the error signal. Execution paths produced with access to ground-truth answers are aligned against failed executions to extract error signals, which are then clustered to reveal systematic rather than incidental failure patterns.
Two gates before any harness update lands. A quality gate filters data leakage and prompt bloat; a performance gate accepts an update only if it improves the current batch without degrading recent batches, with epoch-end held-out validation selecting the best accepted snapshot.
Modular by construction. Execution, evaluation, optimization, and gating are separate agent modules, decoupling the executing agent from the evolutionary pipeline.
Consistent across settings. Gains hold over state-of-the-art baselines across open-domain and enterprise benchmarks, different models, and different agent frameworks, which is the reliability claim the paper is making.
Abstract
Self-evolving agents advance toward autonomy by optimizing their harness---prompts, skills, tools, and execution logic---based on environmental feedback. This paradigm, however, is hampered by three challenges: \textit{credit assignment failure}, where terminal success/failure feedback makes it ambiguous which step caused the error; \textit{shortcut learning}, where agents memorize task-specific patterns rather than acquire generalizable capabilities; and \textit{catastrophic forgetting}, where unguarded updates degrade previously acquired competence. In this paper, we introduce HarnessEvolve, a self-evolving framework that learns from reference trajectories to achieve reliable agent self-evolution. HarnessEvolve decouples the execution agent from the evolutionary pipeline, assigning execution, evaluation, optimization, and gating to independent agent modules, enabling generalizable and stable harness improvements. Specifically, HarnessEvolve overcomes credit assignment failure by generating reference trajectories (execution paths produced when given the ground-truth answers) and aligning failed executions against them to extract error signals, which are clustered to reveal systematic failure patterns. To prevent shortcut learning and catastrophic forgetting, candidate harness updates must pass two gates: a quality gate that filters data leakage and prompt bloat, and a performance gate that accepts each update if it improves on the current batch without degrading recent batches, with epoch-end validation on a held-out set selecting the best-performing accepted agent snapshot. We conduct extensive experiments on several benchmarks spanning open-domain and enterprise scenarios, using different models and agent frameworks. Results demonstrate that HarnessEvolve consistently outperforms state-of-the-art baselines across all benchmarks and settings, confirming reliability across task domains.