Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents
Yujie Feng, Jieping Ye and colleagues at Alibaba show that agents trained by skill-policy co-evolution become dependent on the exact skill bank they trained with, and introduce CODA, an audit that measures how much of the gain survives once skills change after deployment.
Ask this paper
Blind spot. Current evaluations score the co-evolved policy only on its delivered skill bank. In deployment, skills get rewritten, added or removed more often than the policy is retrained.
Cause. The shared acceptance rule (cooperative acceptance) ties the policy's competence to the bank it co-evolved with. The analysis shows this holds regardless of implementation, and sensitivity to drift grows as training proceeds.
CODA protocol. Simulates skill drift and reports worst-case success, a consistency rate under meaning-preserving edits, and a resistance rate to corrupted skills.
Numbers. Across four co-evolution methods and six benchmark settings, worst-case success is 5 to 13 points below the delivered peak, which removes 52% to 93% of the skill bank's gain over the no-skill baseline.
Implication. The target for co-evolution should be using skills however they change, not peak score on the co-evolved bank.
Abstract
Skill-policy co-evolution, in which an agent's policy model and its skill bank improve each other during training, is a promising paradigm for self-improving agents. Current evaluations score the co-evolved policy on the final bank it trained with (the delivered bank). After deployment, however, skills may be rewritten, added or removed more frequently than the policy model is retrained, leaving the policy to operate on a drifted skill bank that neither the training objective nor existing evaluation protocols address. In this paper, we first establish, through theoretical and empirical analysis, that the fragility hidden by this blind spot is not incidental. The acceptance rule these methods share (cooperative acceptance) ties a policy's competence to the bank it co-evolved with, leaving it fragile once skills drift, regardless of the implementation; empirically, sensitivity to skill drift grows as co-evolution training proceeds. We therefore introduce the CO-evolution Drift Audit (CODA), a deployment-oriented evaluation protocol that simulates skill drift and, alongside success rate under worst-case drift, introduces two new metrics: a consistency rate under meaning-preserving edits and a resistance rate to corrupted skills. Across four co-evolution methods and five benchmarks (six settings), worst-case success is 5-13 points below the peak measured for each delivered pair, erasing 52-93% of the delivered bank's gain over the no-skill baseline. CODA thus reframes the target of co-evolution, from peak performance on the co-evolved bank to the ability to harness skills however they evolve, and provides a shared standard against which future methods can be audited.