EvoBelief: Falsifiable World Models for Self-Improving LLM Agents
Janet Wang (Tulane University), Jiarong Jiang, Chandan K. Reddy and colleagues (Amazon) present EvoBelief, a self-evolving world model that stores environment knowledge as falsifiable hypotheses, validates them before use, and narrows or retires them on counterexamples, with the LLM kept frozen.
Ask this paper
Beliefs as a separate axis. Most self-evolving agents improve how they act. EvoBelief targets what the agent believes about environment dynamics, since wrong beliefs propagate across tasks.
Hypothesis lifecycle. Unexpected outcomes and decision-relevant unknowns create hypotheses. Later observations and controlled experiments must validate a rule before it guides plans, and each rule stays linked to its evidence and the decisions it changed.
ALFWorld. Without actor-side transition memory, success improves over a matched frozen world model by 23.9% with Claude Haiku 4.5 and 31.3% with GPT-5.4. Over 80% of decisions under a validated rule occur in a task family other than the one where it was learned.
Drift. On MiniGrid it revises stale rules after unannounced dynamics changes, solving 69.6% of held-out episodes against 1.3% for the frozen model, and 80.8% against 7.1% when color and room names are permuted.
Discovery. In DiscoveryWorld Reactor Lab it completes 83.3% of Normal-tier and 58.3% of Challenge-tier instances, against 0% for each evaluated baseline.
Abstract
Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can fail when their beliefs about environment dynamics are incomplete, incorrect, or outdated. We study a complementary axis of self-evolution: evolving those beliefs during deployment. Such beliefs can support multiple tasks, but incorrect ones can also propagate across them. We introduce EvoBelief, a self-evolving world model that represents environment knowledge as falsifiable hypotheses. Unexpected outcomes and decision-relevant unknowns trigger hypothesis formation. Later observations and controlled experiments provide independent validation before a rule can guide predictions or plans within its supported scope. Counterexamples narrow or retire rules as the environment changes. The language model remains frozen, and each rule remains linked to its evidence and the decisions it changes. Across four benchmarks spanning static dynamics, hidden drift, and scientific discovery, EvoBelief improves task performance and reuses learned rules across tasks. On ALFWorld without actor-side transition memory, it improves success over the matched Frozen World Model by 23.9% and 31.3% with Claude Haiku 4.5 and GPT-5.4, respectively. On MiniGrid, it revises stale rules and recovers after unannounced dynamics changes. In DiscoveryWorld's Reactor Lab, it discovers and validates quantitative laws, completing 83.3% of Normal-tier and 58.3% of Challenge-tier instances, versus 0% for each evaluated baseline. These results demonstrate self-improvement through testable, revisable environment knowledge rather than model-parameter updates.