Does an Agent's History Tell You When Compaction Will Hurt? A Modest, Bounded Effect on the TRACE Paired-Replay Corpus
Egor Pakhomov and Erik Nijkamp (Salesforce AI Research) test whether an agent's recent behavior predicts when context compaction will hurt, using the TRACE corpus of 590 harness-triggered AppWorld compaction boundaries replayed with and without the summary.
Ask this paper
Question. Most long-horizon agents compact on a token budget and ignore what the agent is doing. The paper asks whether pre-boundary history can tell when compaction will cause errors or repeated calls.
Weak signal. Pre-boundary history predicts post-compaction harm only weakly. The best extension trigger reaches held-out AUROC 0.66 against a same-boundary replicate ceiling of 0.72.
Label check. The intuitive has-written label turns out to measure trajectory phase rather than anything specific to writes.
Best trigger. The best frozen, interpretable trigger avoids 21% of harmful boundaries while keeping 84% of compaction opportunities, beating the random-rule expectation on count but not on burden mass.
Open question. Whether such a trigger beats a token-budget rule at matched retention cannot be tested on the release; the authors list what future corpora need to include.
Abstract
Many long-horizon agents compact their context on a global rule, usually a token budget, blind to what the agent was doing. We ask whether the agent's recent behaviour predicts when a compaction will hurt. TRACE's public corpus of 590 harness-triggered AppWorld compaction boundaries replays each boundary from a re-executed prefix state under the pre-compaction context and under the summary, and records the burden of the next actions: calls that error or repeat a call already made. We find that pre-boundary history predicts post-compaction harm only weakly. An internally prespecified contrast by prefix placement is a wide null, and the naive "has-written" label behind it turns out to measure trajectory phase. The best extension-protocol trigger reaches held-out AUROC 0.66 (0.64 on the replicate's own label) against a same-boundary replicate of 0.72; the best frozen, interpretable trigger avoids 21% of harmful (positive-burden) boundaries while keeping 84% of compaction opportunities, and exceeds the random-rule expectation on count but not on burden mass (a post hoc comparison). Whether the best trigger beats a token-budget rule at matched retention cannot be evaluated on the release. We state what corpora should ship to answer it.