Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Haoyu Zhao, Haitham Bou-Ammar, Jun Wang and colleagues (University College London, Huawei Noah's Ark Lab UK, University of Liverpool) present Memento 3, in which a frozen LLM agent learns an explicit world model as a natural-language rulebook, compiles it into executable code, and accepts a revision only when replay reproduces every observed transition.
Ask this paper
Rulebook as memory. The agent keeps revisable hypotheses about environment dynamics in a natural-language rulebook and leaves unknown aspects underspecified, so limited observations do not force a premature commitment to one world model.
Verified compilation. Each rulebook revision is compiled to code for prediction and planning. The code is accepted only if an LLM judges it faithful to the rulebook and cell-exact replay reproduces the observed transitions.
ARC-AGI-3. The single-model agent clears every level of all 25 public games, reaches the RHAE ceiling of 100.0, and uses 7,518 actions, which is 44% of the human action count.
Population extension. Several world models are maintained in parallel, share interaction evidence, and use their disagreeing predictions to choose where to explore.
Pong. In an Atari Pong case study, a controller derived from the learned model wins 21:0 in three episodes with different openings and makes no further LLM calls.
Abstract
Learning to act in unfamiliar environments requires agents to infer how the world works and revise that understanding as new evidence arrives. Yet limited observations can support multiple world models that explain past interactions but predict different outcomes in unseen states. We introduce Memento 3, building on the Memento series to enable frozen LLM agents to continually learn explicit world models through external memory. The agent maintains a natural-language rulebook as persistent semantic memory, recording revisable hypotheses about environment dynamics while leaving unknown aspects underspecified. It compiles this rulebook into executable code for prediction and planning. Through a continual loop of observation, reflection, rule revision, compilation, and verification, the agent uses prediction errors to refine both the rulebook and its code. Updated code is accepted only when the LLM judges it faithful to the rulebook and cell-exact replay reproduces the observed transitions. We investigate this process as a model-based route to recursive self-improvement (RSI): the agent autonomously explores the environment, revises its world model, and uses verified updates to guide subsequent interaction and learning, while the underlying LLM remains fixed. A population extension maintains multiple world models in parallel, sharing interaction evidence and using their predictions to guide exploration. On ARC-AGI-3, the single-model agent clears every level of all 25 public games, achieves a mean Relative Human Action Efficiency (RHAE) of 100.0, and uses 44% of the human action count. In an Atari Pong case study, a learned feedback controller wins 21:0 in each of three evaluated episodes with different openings, without further LLM calls.