Scaling Discovery Requires Forgetting
Shangyin Tan, Ben Zhou, Lakshya A Agrawal, Alex Dimakis, Matei Zaharia and colleagues (UC Berkeley, Stanford, Bespoke Labs) show that automated-discovery agents plateau as they accumulate history, and that periodically discarding all sessions and search state, keeping only the best candidate, lets the same budget go further.
Ask this paper
Plateau. Running one agent longer, running agents in parallel, and letting parallel agents share candidates all plateau. Each agent anchors on its own past attempts and makes fewer and smaller edits.
Forgetting. Periodically drop every agent's session and the candidate history, then restart from the best candidate found so far.
Tasks. Four tasks across heuristic programming, kernel optimization and model training, including an AtCoder heuristic contest problem and Anthropic's performance take-home.
Results. Forgetting with a communicating team gives the best result on all four tasks, up to 15% better than parallel agents and up to 39% better than a single agent at the same budget.
Open question. When to forget and what to keep are left for future work.
Abstract
Automated research with LLM agents has shown promising results in various domains. Through iterative refinement and scored verifiers, LLM agents are capable of discovering new ideas. There are different ways to scale automated research: let one agent iterate for longer, run several agents in parallel, or further let the parallel agents communicate and build on one another's candidates. However, we find that all three strategies eventually plateau as the agents accumulate history. Each agent becomes anchored to its own past attempts, making fewer and smaller edits. Beyond this point, additional budget brings little further improvement. In this paper, we show that the key to scaling discovery further is to forget. Periodically discarding the search state, including every agent's session and the candidate history, and restarting from the best candidate found so far, unblocks the search and pushes the same budget further. We implement forgetting in a team of agents with communication and evaluate it on four tasks spanning heuristic programming, kernel optimization and model training. The results show up to 15% improvement compared to parallel agents, and up to 39% compared to a single agent. Our results suggest that, for open-ended discovery, what an agent is allowed to remember matters as much as how much compute it is given.