From AutoResearch to Continual AutoResearch: Do Agents Learn Across Tasks?
Jeremy Qin, Leo Feng and colleagues at RBC Borealis and the ELLIS Institute Tübingen introduce Continual AutoResearch, an environment that compares persistent and reset instances of the same research agent on matched task streams to test whether agents carry insight across tasks.
Ask this paper
Environment. Nine task families covering synthetic tasks, symbolic regression, time-series forecasting and tabular AutoML, with matched data and time budgets.
Agents. Five frontier CLI agents are measured on first-solution quality, research efficiency and final held-out performance.
Efficiency gain. Persistent state improves research efficiency for all five agents: they reach valid, strong solutions sooner.
No final gain. These process gains do not consistently produce better final solutions. Once first-solution reward is subtracted, no agent keeps an advantage.
Memory ablation. Varying what the agent retains shows that more history is not automatically helpful. Retaining experience and using it well are separate capabilities.
Abstract
Autoresearch has shown that agents can autonomously write code, run experiments, and iteratively improve machine learning solutions. Yet most evaluations start a new agent session from scratch for each new problem, making them costly and leaving unclear whether agents can retain and apply prior insights when adapting to new tasks. We introduce CONTINUAL AUTORESEARCH, an open-ended environment that compares persistent and reset instances of the same agent under matched task streams, data, and time budget. Continual AutoResearch covers nine task families, spanning synthetic tasks, symbolic regression, time-series forecasting, and tabular AutoML. Across five frontier agents, we measure first-solution quality, research efficiency, and final held-out performance. To distinguish insight transfer from session continuity, we run a memory ablation that vary what information agents retain from prior tasks. We show that persistent state improves research efficiency of all five agents, allowing them to find stronger solutions sooner. However, these gains do not consistently translate into better final solutions. Overall, our findings suggest that retaining experience and utilizing it effectively are distinct capabilities.