SIMLIFE: Pattern Understanding for Long-Horizon Human-Agent Partnership
Run Peng and colleagues build SimLife, a simulator of long-term household life with visual observations, ground-truth action logs and synthetic dialogue, and SimLife-BP, which tests whether a model can infer latent behavioural rules from weeks of observation.
Ask this paper
106 episodes averaging 15.49 hours and 38.57 in-game days. With 1,439 question-answer pairs probing direct, counterfactual, noisy and inverse reasoning under different levels of rule hinting.
Models predict without understanding the rule. Frontier models often reach surface-level prediction accuracy while failing to recover the if-then rule that generated the behaviour.
Frequency heuristics substitute for reasoning. Models fall back on how often something occurred rather than reasoning over evidence, which is why they break when a pattern changes.
Adaptation is the weakest axis. Performance drops when behavioural patterns shift mid-episode, which is the case a household assistant would actually face.
Abstract
Understanding humans over long horizons requires agents to infer not only what people need in the moment, but also how routines form, why they repeat, and when they change. We introduce SimLife, a scalable platform for simulating long-term household life with rich visual observations, ground-truth action logs, and synthetic dialogues with audio. Built on SimLife, SimLife-BP evaluates long-context pattern understanding: the ability to infer latent behavioral rules from weeks or months of everyday observations. The benchmark contains 106 episodes averaging 15.49 hours and 38.57 in-game days, and 1,439 question-answer pairs. Each task probes direct, counterfactual, noisy, and inverse reasoning under different levels of rule hints. Evaluating frontier models and architectures, we find that current models often achieve surface-level prediction without comprehensive rule understanding, rely on frequency-based heuristics rather than if-then reasoning over evidence, and struggle to adapt when behavioral patterns change. These findings suggest that long-context pattern understanding remains a major bottleneck for future embodied agents, while SimLife opens a broader space for studying memory, personalization, adaptation, and long-horizon planning in everyday human-AI interaction.