Agent Plasticity: Measuring Self-Improvement Through Experience
Harman Singh, Anirudh Goyal, Jason Weston, Gabriel Synnaeve, Rob Fergus and colleagues at Meta Superintelligence Labs (with UC Berkeley and Princeton's Sanjeev Arora) propose agent plasticity, a measure of how efficiently an agent turns experience into gains on held-out tasks.
Ask this paper
Protocol. Model weights stay frozen. Each episode starts in a fresh context, and later instances inherit the notes, skills, Python tools and memory that earlier instances wrote into CLAUDE.md, AGENTS.md and SKILL.md files.
Metric. Plasticity is the held-out in-distribution score gain up to an estimated saturation point, divided by the learning cost spent to reach it. Training, held-out ID and held-out OOD (stronger opponents) splits are scored separately at every checkpoint.
Divergent trajectories. In chess, Go, Hex and NetHack, frontier models given the same opportunities improve very differently. Claude Fable 5 reaches the highest fitted endpoint, while GPT-5.6 Sol has the highest plasticity per dollar and overtakes Claude Opus 4.8 despite starting lower.
NetHack. Only Claude Opus 5.5 shows a significant rise over 40 checkpoints: a fitted gain of 66.25 normalized points for $1,072.73 of learning cost. Gains inside the training regime often transfer only partly to OOD conditions.
Where loops break. Low-plasticity agents often fail to reuse a relevant artifact that already exists. High-plasticity agents reuse artifacts and still fail, which points to artifact quality, generalization or application as the limit.
Abstract
AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns. Evaluating self-improvement requires answering three questions: does future performance improve and generalize beyond the interactions that enabled learning; how efficiently are new capabilities acquired; and where does the self-improvement process break down? To answer these questions, we study self-improvement in a controlled setting where agents amortize past experience into reusable artifacts that are inherited by future instances. At each checkpoint, we measure performance on training and held-out environment interactions while accounting for learning cost. We introduce agent plasticity, the efficiency with which an agent converts experience into gains in future held-out performance. Across multiple environments, frontier models exhibit sharply different improvement trajectories despite comparable opportunities to learn. Some achieve substantial and persistent gains, while others remain near or below their initial performance, and gains within the training regime often transfer only partially to out-of-distribution conditions. Endpoint capability and acquisition efficiency also diverge: the agent that ultimately performs best need not be the one that improves most efficiently. Tracing failures through the improvement loop further reveals different candidate bottlenecks. Agents with low plasticity often fail to reuse relevant artifacts, whereas more plastic agents may still fail despite reusing relevant artifacts, pointing to limitations in artifact quality, generalization, or application. Evaluating self-improving agents requires measuring not only what they can do, but how effectively they become better through experience.