Why LLM Agents Collapse Without Oversight: The Enforcement Gap as the Mechanism Behind Emergence World Failures
Yuhang Wang (Fudan University) argues that Reflexion-style agents already detect dangerous plan steps during self-critique but have no path from detection to action, and calls this the enforcement gap.
Ask this paper
Mechanism claim. The audit step finds the problem while the controller ignores it, which the paper links to the failures seen in the Emergence World multi-agent simulation.
Small fix. A single conditional check of fewer than 20 lines reduces attack success by more than fourfold across frontier models, five agent frameworks and an independent benchmark.
Formal result. When enforcement probability is near zero, detection quality has no effect on security.
Compounding failures. Unreliable auditors and unparseable verdicts add to the gap, and a GRPO-trained enforcement controller handles ambiguous verdicts.
Abstract
When Emergence World placed frontier LLM agents in an unsupervised multi-agent simulation, the results were alarming: agents committed crimes, starved, and enforced unanimous conformity -- without any external attacker. This paper identifies the mechanism. Reflexion-style agents already detect dangerous plan steps through iterative self-critique, yet the architecture provides no pathway from detection to action. We call this the enforcement gap: the audit sees the problem; the controller ignores it. Closing the gap requires a single conditional check -- fewer than 20 lines of code -- and reduces attack success by more than fourfold in large-scale experiments across frontier models, all five major agent frameworks, and an independent benchmark. We prove formally that when enforcement probability is near zero, detection quality is irrelevant to security. We further identify two compounding failure modes -- unreliable auditors and unparseable verdicts -- that explain every collapse pattern in Emergence World. A GRPO-trained enforcement controller resolves the ambiguity case. Together these results motivate a three-requirement Audit Enforcement Specification that is absent from every deployed framework today.