The Right Memory in the Wrong Context: Verifying Retrieval Admissibility in Long-Term Agent Memory
Zi Wang, Xiaowei Xu and colleagues (University of Arkansas at Little Rock) argue that long-term agent memory needs an admissibility check separate from relevance, and build a verification framework that labels each memory-query pair admissible, inadmissible or unresolved.
Ask this paper
Problem. A memory can be relevant and still belong to another user, violate policy, or be in the wrong lifecycle state. Recall and final-answer accuracy do not reveal this, since a route can look safe because it missed evidence, and a correct answer can follow exposure to inadmissible memory.
Namespace filtering. A post-hoc top-20 reanalysis of RHELM and MemOps (3,767 queries) raises anchor recall from 0.432 to 0.533 and 80%-recall feasibility from 0.237 to 0.311, while cutting exact similarity evaluations by 98.3%.
Text-only verifiers fail. In a 72-case diagnostic, a reference that uses released metadata preserves required evidence, while neither text-only verifier detects violations under a 1% false-denial limit on required anchors.
Exposure and disclosure. In 16 controlled exposure scenarios, only one of four readers shows a disclosure increase whose 95% CI excludes zero (+0.156). The authors treat exposure and disclosure as separate stages to verify.
Reporting discipline. Each stage is evaluated on a separate population, and the 1,523-case accuracy comparison (+0.053 to +0.068) is labeled observational because recall also changes.
Abstract
Long-term-memory agents can retrieve relevant information that is inadmissible for the current request because it belongs to another principal, violates policy, or reflects an incompatible lifecycle state. Recall and final-answer accuracy do not reveal this: a route can appear safe by missing required evidence, while a correct answer may follow inadmissible prompt exposure. We introduce a retrieval-admissibility verification framework that assigns each memory-query pair one of three statuses (admissible, inadmissible, or unresolved), compares routes at matched required-evidence recall with bounds for unresolved cases, and tracks memory IDs through prompt exposure while linking exposure to target-level disclosure. We evaluate its stages on separate, non-pooled populations. A post-hoc top-20 reanalysis of frozen rankings from two public long-term-memory benchmarks, RHELM and MemOps, covers 3,767 queries. All released anchors lie within trusted query namespaces; with within-namespace scores unchanged, off-namespace filtering cannot lower their ranks. Top-20 anchor recall increases from 0.432 to 0.533, 80% recall feasibility from 0.237 to 0.311, and exact similarity evaluations decrease by 98.3%. In a frozen 72-case development diagnostic, a released-metadata reference preserves required evidence, whereas neither text-only verifier detects violations under the 1% required-anchor false-denial limit. Across 1,523 paired benchmark-native cases, namespace routing is associated with judged-accuracy gains of 0.053-0.068 across three readers; recall also changes, so this comparison is observational. In 16 controlled exposure scenarios, only one of four reader-specific 95% confidence intervals excludes zero for relevant-inadmissible literal disclosure (+0.156, 95% CI [0.031, 0.312]). Results motivate separate verification of candidate support, admissibility, prompt exposure, and answer disclosure.