When Do Summaries Become Instructions? Unsupported Policy in LLM Agents' Self-Written Handoff Summaries
Wenhui Chu at the University at Albany audits the handoff summaries that long-running agents write when compacting context, checking whether they invent permissions or restrictions and whether the next agent instance acts on them.
Ask this paper
Setup. Synthetic tool-use tasks carry a ground-truth permission ledger. Same-snapshot replay reruns the successor with suspect sentences replaced by neutral filler, which isolates their effect.
Frequency. Unsupported restrictions appeared in 1.2% and 2.4% of readable summaries from two open-weight models, and unsupported authorizations in 0.1% and 1.3%.
Effect of false approvals. Successors given a false approval acted without permission in 11 to 16 of 60 runs per held-out set, against none in the main sets when given the full history. Removing the sentence cut unauthorized actions from 24 to 7 of 120 runs (post hoc).
Restrictions. The pre-specified replay test for restrictions was not significant (p = 0.58). Most invented restrictions trace back to the agent's own earlier reasoning that an action needed approval.
Fix. Replacing the system prompt sentence "Some actions need approval." with one stating where requirements come from cut unsupported restrictions from 5.0% to 1.8% (development) and 1.8% to 0.3% (held-out), both p < 10^-9.
Abstract
Long-running LLM agents compact their context, replacing most history with a summary. We ask whether handoff summaries state permissions or requirements the record does not support, whether such sentences change what the next instance (the successor) does, and where they come from. Synthetic tool-use tasks carry a ground-truth permission ledger; same-snapshot replay reruns the successor with those whole sentences replaced by neutral filler. Two AI readers, reading only rule-flagged held-out summaries, confirmed unsupported restrictions in 1.2% and 2.4% of all readable summaries from two open-weight models, and unsupported authorizations in 0.1% and 1.3%. For restrictions, a pre-specified replay test was not significant (p = 0.58); a follow-up, planned after it, found a fragile effect (p = 0.027). Neutralizing false approvals cut unauthorized actions from 24 to 7 of 120 held-out runs (post hoc; later set: 17 to 11 of 48). Most development restrictions followed the agent's earlier reasoning that an action needs approval (post hoc). In pre-specified tests, replacing the system prompt's "Some actions need approval." with one saying where requirements come from cut unsupported restrictions from 5.0% to 1.8% of development summaries and, on held-out templates, from 1.8% to 0.3% (both p < 10^-9), while requests for approvals the ledger requires barely changed. Code and data: https://github.com/Chuw1/handoff-audit