Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents
Theo Rusu, Sourena Khanzadeh and Manar Alalfi (Toronto Metropolitan) test the assumption that graph-structured memory beats flat retrieval, find it does not at matched budget, and then show forgetting is nearly free.
Ask this paper
The negative result is the contribution: At a matched candidate-generation budget of five retrieval roots on LongMemEval, the graph gets token F1 0.417 against a flat vector baseline's 0.468, with a paired bootstrap over 500 questions giving -0.050 (95% CI -0.085 to -0.016).
Where the graph loses: Questions requiring recall of a specific prior assistant turn: judged correctness falls from 0.911 to 0.607. Decomposing a turn into entities discards the surface form those questions depend on.
Forgetting works: One pruning pass over a persistent 27,021-node graph, scoring on recency, access frequency, degree centrality and age, removes 9.8% of nodes and 9.5% of bytes with token F1 unchanged and judged correctness down 1.6 points, bounded at 3.8.
Honest scoping: The authors state the result characterizes this extraction-based pipeline with a single small extractor on one benchmark, not graph memory in general. That caveat is doing real work.
Why it belongs next to Agent Zero Memory: This is the empirical case for why committing to a single organizing structure inherits its blind spots.
Abstract
Knowledge graphs have been proposed as a structured alternative to flat retrieval-augmented generation for long-term agent memory, on the assumption that representing conversations as entities and relations improves recall. We evaluate that assumption directly. Our framework extracts each conversational turn into typed nodes and attributed edges, answers questions from a two-hop subgraph, and periodically prunes nodes that score low on a weighted combination of recency, access frequency, degree centrality, and age. On LongMemEval, the graph does not outperform a flat vector baseline at a matched candidate-generation budget of five retrieval roots: token F1 is $0.417$ against $0.468$, and a paired bootstrap over 500 questions gives $Δ= -0.050$ (95\% CI $[-0.085, -0.016]$). The gap is widest on questions that require recalling a specific prior assistant turn, where judged correctness falls from $0.911$ to $0.607$, suggesting that decomposing a turn into entities discards the surface form these questions depend on. The forgetting module is more successful. Applied once to a persistent 27{,}021-node graph, it removes 9.8\% of nodes and 9.5\% of stored bytes; token F1 is unchanged ($+0.001$, 95\% CI $[-0.015, +0.016]$) and judged correctness falls by $1.6$ points, with the 95\% interval bounding any loss at $3.8$ points ($[-0.038, +0.006]$). Because our extractor is a single small model evaluated on one benchmark, these results characterise this extraction-based pipeline rather than graph-structured memory in general. Code: https://github.com/skhanzad/Selective-Amnesia