← All papers  /  Nov 17, 2026
Agents

Do LLM Agents Still Need to Be Told How to Self-Improve?

First page
Do LLM Agents Still Need to Be Told How to Self-Improve?
The curator’s take

Shobhita Sundaram (MIT CSAIL), Cansu Sancaktar (University of Tübingen) and Phillip Isola (MIT CSAIL) give a frozen LLM agent a compute budget and basic tools, with no prescribed learning loop, and find that capable models design their own self-improvement procedure.

Ask this paper

Key points
01

Setup. A teacher instance of the model can write games, watch a player instance of the same model play them, and edit the player text context such as prompts and skill files. It gets no real tasks, no validation scores and no instructions on when to use each tool.

02

Capable models self-improve. Sufficiently capable models build an effective learning procedure and improve the player on held-out games with undisclosed mechanics.

03

Shared strategies. Models from different families converge on the same methods: games with calibrated difficulty, controlled A/B comparisons and their own validation suites. They also spend more effort on harder tasks, as in asymmetric self-play.

04

Transfer. The learned text files improve play on external benchmarks with different game horizons, mechanics and 3D navigation.

05

Safety reading. Leaving out an explicit self-improvement loop does not prevent a model from self-improving when it has the tools.

Abstract

LLM agents increasingly drive their own learning. Self-improvement methods delegate various components of model optimization to the models themselves. As models become more capable, this motivates a basic question: do these increasingly capable models still need us to specify how they should self-improve? In this paper we study an autonomous paradigm in which, given basic primitives, resources, and a general goal for self-improvement, agents construct their own self-learning algorithm for context-based optimization. Using the setting of gameplay as a testbed, a frozen LLM agent is given a compute budget and tools for collecting experience and updating text documents, and allowed to freely experiment without an externally imposed loop or access to real-world data. We find that sufficiently capable models do self-improve under these conditions, demonstrating strategies that align with classic cognitive frameworks for self-regulated learning and constructs from machine learning literature on self-generated curricula. Furthermore, we study the extent to which the learned text artifacts transfer to external benchmarks, observing improvement across different game horizons, mechanics, and 3D navigation.

Every Monday
Get next week’s papers.
Subscribe on Substack