MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling
Renhao Li, Dayiheng Liu, Min Yang and colleagues (University of Macau, SIAT-CAS and Alibaba's Qwen team) train experience learning, the ability to turn past interaction feedback into reusable knowledge, as a general model capability instead of an external scaffold, by fine-tuning Qwen3.5-122B-A10B on trajectories from 1,000 verifiable task families built to share structure across tasks.
Ask this paper
Task-family design. Each family keeps one reusable factor fixed and varies the task-specific details, so a model can transfer what it learned without copying an earlier solution. Shared-Tool families hold black-box tools fixed to teach operational experience; Shared-Workflow families hold a latent procedure fixed to teach procedural experience.
Training data. The authors scale environments under both designs, collect sequential interaction trajectories, filter them, and fine-tune on 4,956 trajectories drawn from 1,000 families.
Results with frozen weights. After training, parameters stay frozen and the model improves only through accumulated experience at deployment. It gains +8.5 points on SpreadsheetBench, peaks at +2.3 on APEX-Agents and gains +14.8 on AutomationBench.
Transfer check. The gains persist when the downstream domains are excluded from training, which supports the claim that the model learned how to learn from experience and did not memorize domain content.
Why it matters. Most experience-reuse methods tune prompts, memory stores or scaffolds around a fixed model. This paper moves the capability into the weights, which makes it portable across harnesses.
Abstract
Language-model agents can accumulate interaction feedback, but reliably turning that feedback into reusable experience remains challenging. Existing methods either optimize external experience scaffolds or train experience-related behavior on specific task distributions, leaving open whether experience learning itself can become a transferable model capability. Training such a capability calls for supervision beyond isolated tasks, which mainly support in-episode problem solving rather than revealing what knowledge remains useful across episodes. In this work, we introduce MetaExp, a scalable framework that trains this capability through verifiable task families with controlled cross-task structure. Each family preserves a reusable factor while varying task-specific realizations, enabling transfer without direct solution reuse. We propose two complementary designs: Shared-Tool families keep black-box tools fixed across tasks to elicit operational experience, while Shared-Workflow families preserve latent procedures across changing realizations to elicit procedural experience. We scale environments under both designs, collect and filter sequential interaction trajectories, and fine-tune Qwen3.5-122B-A10B on 4,956 trajectories from 1,000 task families. With parameters frozen during downstream evolution, MetaExp improves by +8.5 points on SpreadsheetBench, peaks at +2.3 on APEX-Agents, and improves by +14.8 on AutomationBench. The gains persist under domain-excluded training, supporting experience learning as a transferable capability for continued non-parametric improvement after deployment.