Self-Evolving Coding Agents via Harness-Policy Optimization
Zhepei Wei, Wenlin Yao, Baolin Peng, Hao Cheng, Jianfeng Gao and colleagues (Microsoft Research, University of Virginia, CMU) propose Self-HPO, which alternates harness optimization and policy RL for a coding agent and uses the current policy as the proposer of its own harness changes.
Ask this paper
Core argument. Harness and policy depend on each other. A harness tuned for one checkpoint stops targeting the failure modes of the next one, and RL under a fixed harness only sees the trajectories that harness elicits.
Loop. In the H-step the policy analyzes its own failures and proposes harness candidates. In the P-step the policy is trained with RL under the selected harness. Tasks that stay hard during training become the next harness search set.
Results. On three coding benchmarks, Qwen3.5-35B-A3B's average resolve rate rises from 57.8% to 64.8%, ahead of harness-only and policy-only baselines, and the gains carry to out-of-distribution settings.
Superadditive gain. Each optimized component keeps its gain when paired with the base version of the other, and the joint gain exceeds the sum of the two individual gains. Pass@1 and Pass@k both improve.
Stronger proposer. Using Opus 4.8 as the external harness proposer in the same framework raises performance further, so the method also works as a recipe for distilling a frontier model's harness judgment into a smaller policy.
Abstract
Large language model (LLM) agents have shown promising potential in coding tasks, and their performance is jointly determined by the policy model and the harness that mediates their interaction with the environment. Indeed, harness and policy are mutually reinforcing and evolving only one component is suboptimal---a harness tuned for one checkpoint may not target the failure modes of an updated policy, while policy learning remains limited to the trajectories elicited by a fixed harness. In light of this, we propose Self-HPO, a recursive self-improving framework for coding agents that alternates harness and policy optimization, using the evolving policy as its own harness proposer. Specifically, in each round, the current policy first analyzes its own failures and proposes new harness candidates (H-step). The policy is then updated through reinforcement learning (RL) under the selected harness (P-step), and hard tasks identified during training form the next harness search set, keeping harness optimization targeted to the policy's evolving failure modes. Across three coding benchmarks, Self-HPO consistently outperforms harness-only and policy-only baselines, improving Qwen3.5-35B-A3B's average resolve rate from 57.8% to 64.8%, with gains generalizing to out-of-distribution settings. Further analysis shows that both optimized components retain gains even when paired with their base counterparts, while their joint gain exceeds the sum of their individual gains. Self-HPO also improves reliability (Pass@1) and expands the capability frontier (Pass@k) over the base agent. Additionally, we show that with a stronger external model (e.g., Opus 4.8) as the harness proposer, the same framework yields a further performance boost.