Measuring the Value of World-Model Updates: A Counterfactual Utility Protocol for Continual Adaptation
Anqi Peter Li (Substrate Labs) and Kaden Kim (UC Berkeley) introduce the fork ledger, which measures whether an individual world-model update helped by running matched update and hold branches from the same point of a deployment stream.
Ask this paper
Problem: Replay schedules and prediction-error triggers decide when to update a continual world model, but one deployment run cannot show how the same model would have performed had it kept its parameters.
Fork ledger: At pre-registered decision points the stream branches into update and hold continuations under common random numbers, and the return difference between them is recorded on identical episodes.
Results: Always applying one fixed update mechanism lowers return on all three tasks. CartPole loses 144.0 (95% CI -185.4 to -116.1, against a converged return near 650), Walker 82.8 and Cheetah 18.6.
Robustness: Excluding the 27 of 720 forks that collapsed, CartPole and Walker keep their sign and Cheetah becomes unresolved. Each task contributes 240 forks over five checkpoints and two drift directions.
Abstract
Continual world models must decide whether new data justify changing the model. Fixed replay schedules and prediction-error triggers specify when to update, but neither reveals the value of an individual update: one deployment run cannot show how the same model would have performed at that moment had it held its parameters. We introduce the fork ledger, which branches a deployment stream at pre-registered decision points into matched update and hold continuations under common random numbers. It evaluates both continuations on the same episodes and records $ΔR = R_{\mathrm{update}} - R_{\mathrm{hold}}$. Always applying one fixed update mechanism lowers return on all three simulated control tasks: CartPole ($-144.0$; checkpoint-bootstrap $95\%$ CI $[-185.4,-116.1]$, against a converged return near $650$), Walker ($-82.8$; $[-101.1,-61.7]$) and Cheetah ($-18.6$; $[-29.0,-6.6]$). Divergence is an outcome of applying the update, so the estimand counts every attempted fork; restricted to the $693$ of $720$ that did not collapse, CartPole and Walker are unchanged in sign ($-113.4$ and $-82.1$) and Cheetah becomes unresolved ($-3.9$; $[-17.5,+13.0]$). The task is the unit of inference: each contributes $240$ attempted forks over five pretrained checkpoints crossed with two drift directions. The ledger makes counterfactual utility observable for a fixed mechanism, allowing triggers to be judged by the updates they select rather than by surprise detection alone.