Prune the Summary, Not the History: Source-Judged Compaction for Long-Horizon Agents
Guanghui Min and Chen Chen (University of Virginia) show that context-compaction summaries mix verified facts with the compressor's own plans, trace most agent failures to those unsupported entries, and remove them with a cheap source check.
Ask this paper
Diagnosis. Compaction prompts ask for decisions and next steps, so summaries contain both environment facts and compressor reasoning. The agent cannot tell them apart later.
Failure attribution. With a widely used structured compressor, 83% of failed runs trace back to an action that executes a specific summary entry, and most of those entries have no support in any environment output.
Source-Judged Compaction. A training-free post-processing step scores each summary entry for whether it has a source in environment outputs or the previous summary, and deletes those that do not.
Cost. It changes neither agent nor compressor and adds one judge call per compaction. The judge is Jev, a non-generative decision model that returns a calibrated probability.
Result. On AppWorld and OfficeBench with two agent models, cross-run reliability rises 2.1% to 8.9% with no increase in agent tokens.
Abstract
Long-horizon LLM agents rely on context compaction: when the interaction history outgrows the context window, a compressor replaces it with a natural-language summary. We show that these summaries contain more than a record of what happened. Compaction prompts ask for decisions and next steps, so each summary mixes facts established by the environment with reasoning written by the compressor, and the agent cannot tell the two apart afterwards. In an analysis of a widely used structured compressor, 83% of failed runs trace back to an action that carries out a specific summary entry, most of which no environment output supports. We propose Source-Judged Compaction (SJC), a training-free post-processing step that scores each summary entry for whether it has a source in the environment outputs or the previous summary, and removes the entries that do not. It changes neither the agent nor the compressor and adds one inexpensive judge request per compaction. The judge is Jev, a non-generative decision model that answers the source question with a calibrated probability instead of text. On AppWorld and OfficeBench with two agent models, SJC improves cross-run reliability over the underlying compressor by 2.1%-8.9% without increasing the agent's token usage.