Predict Before You Modify: Learning Improvement Dynamics for Self-Evolving Agents
Zixuan Xie (University of Virginia), Chang Xu and Li Zhao (Microsoft Research) and Yuhua Jiang (Tsinghua) add an editable Improvement Dynamics Model (IDM) to self-evolving agents that predicts how a proposed modification will change task outcomes before it is evaluated.
Ask this paper
Problem. Self-evolving agents only learn whether a modification helped after a costly evaluation, and aggregate scores hide which tasks failed for which reason.
Predict, then revise. Before evaluation, the IDM predicts task-level outcomes of a proposed change and can trigger a revision. After evaluation, the observed outcomes update the IDM, so predictive knowledge accumulates across generations.
Results. Built on HyperAgents, it beats HyperAgents in multilingual coding, paper review, robotics reward design and Olympiad grading, by up to 46% in average performance over generations. Under matched token budgets median AUBC stays 10.9% higher.
Held-out and transfer. On 166 held-out coding tasks, selected agents gain 13.25 points over the initial agent, against 5.42 for HyperAgents. An IDM transferred from another domain adds a further 4 points on held-out grading tasks.
Abstract
Self-evolving agents improve through repeated cycles of modification and evaluation. However, existing methods receive feedback on modifications only after costly evaluations, and aggregate scores may obscure distinctions between failures that require different repairs. We introduce predictive self-evolution through an editable Improvement Dynamics Model (IDM). Before evaluation, the IDM predicts task-level outcomes of a proposed modification and guides an optional revision. After evaluation, past modifications and their observed outcomes are used to improve the IDM, allowing predictive knowledge to accumulate across generations. The system can thus improve how it solves tasks, proposes modifications, and predicts their effects. Building on HyperAgents, we evaluate this approach across multilingual coding, paper review, robotics reward design, and Olympiad-level mathematical grading. Our approach outperforms HyperAgents across all four domains, with up to 46% higher average performance over generations. Gains also persist under matched token budgets that include both agent evolution and task evaluation, and extend to held-out tasks. Forecast accuracy improves as the IDM evolves, and the learned predictive knowledge transfers to improve evolution in a new domain.