Compact-Memory LLM Agents via Online Max-Member Clustering and Atom-Aware Packing
Jiahe Geng, Jinpeng Wang and Kun Yuan build RSM-full, an online clustered-memory pipeline for LLM agents operating under a 2k to 5k prompt-token budget, and show the gain comes from how memories are merged and packed rather than from raw recall.
Ask this paper
Two design choices. A cosine-gated max-member merge rule for writing memories, plus an atom-aware grouped packer that assembles retrieved content into the prompt.
Quality per token. On AMA-Bench, RSM-full reaches 83% of full-context quality at 32% of the token cost at a 4k budget, and beats Online K-Means by 3.5 to 6.0 points across the 2.6k to 5k regime (p<.001).
Ablations isolate the two parts. The merge rule contributes 5.7 points over Online K-Means and matched-tau DP-means; the grouped packer contributes 5.0 points over flat concatenation.
Reproduces on a second benchmark. On RealMem the method beats Budget-RAG by 0.69 points, Streaming-Proto by 2.97, and the A-MEM agentic-memory baseline by 1.65; it is on par with BM25-RAG rather than better.
Scope stated honestly. The authors mark the method as a Pareto point for tight budgets and note that higher-token baselines remain stronger outside it.
Abstract
Many long-horizon LLM deployments face tight prompt budgets: latency, cost, and context limits make full-context prompting impractical as interaction length grows. The key question is then not raw recall alone, but which memory design gives the best quality--token trade-off in the compact-memory regime. We present \textbf{RSM-full}, an online clustered-memory pipeline designed for a strong quality--token Pareto point. RSM-full combines two design choices: a cosine-gated \emph{max-member merge} write rule and an atom-aware grouped context packer. On AMA-Bench, our primary compact-memory benchmark, it reaches $83%$ of Full-Context quality at $32%$ of the token cost at a $4$k budget; under four-seed averaging it beats the closest streaming-clustered baseline (Online K-Means) by $+3.5$--$6.0$,pp ($p{<}.001$) across the whole ${\sim}2.6$k--${\sim}5$k regime. Three-seed ablations show most of this gain comes from the merge rule ($+5.7$,pp over Online K-Means and matched-$τ$ DP-means) and the grouped packer ($+5.0$,pp over flat concatenation). The pattern reproduces on RealMem, an independent long-horizon persona-memory benchmark: RSM-full improves on Budget-RAG ($+0.69$,pp, $p{=}.006$), is on par with BM25-RAG (paired $Δ{=}{+}0.27$,pp, $p{=}.47$; we do \emph{not} claim BM25 equivalence in the equivalence-test sense), and significantly outperforms Streaming-Proto ($+2.97$,pp) and the closest reproduced 2025 agentic-memory baseline A-MEM ($+1.65$,pp, $p{<}.001$). Across benchmarks the message is consistent: under tight budgets, compact-memory performance is driven mainly by how streaming memories are merged and how retrieved content is assembled. Overall, RSM-full is most useful when answeroughly $2k$--$5k$ prompt tokens, where itdefines a strong compact-memory Pareto point; higher-token baselines remain stronger outside this regime.