MemCredit: From Relevance to Utility for Decision-Time Memory in Long-Horizon Web Agents
Quanjiang Guo, Sijie Wang and colleagues at UESTC and Nanyang Technological University introduce MemCredit, which picks agent memories by their measured effect on the next action rather than by semantic relevance.
Ask this paper
Method. Counterfactual interventions measure how much each memory changes the agent decision in a given state. These credits train a lightweight utility estimator that ranks candidates retrieved by relevance.
ALFWorld. Under a 500-token memory budget, MemCredit reaches 86.57% SR@50 against 65.67% for BM25, and 90.30% at 750 tokens.
Efficiency. It uses 152.1 memory tokens per decision, 64.4% fewer than MemArbiter and 74.5% fewer than BM25, without changing the agent policy.
Analysis. Memory utility is sparse and state dependent: most retrieved memories have zero credit, and semantic relevance correlates poorly with utility.
Abstract
Large language model (LLM) agents increasingly operate over long Web interaction histories, making memory management critical for reliable and efficient decision making. Existing methods primarily optimize storage, retrieval, or compression, but the memories most semantically relevant to the current state are not necessarily those that most improve the agent's action. We introduce MemCredit, a decision-centric memory framework that shifts memory selection from semantic relevance to decision utility. MemCredit estimates the state-, reference-context-, and policy-conditional marginal contribution of individual memories through controlled counterfactual interventions, and distills these intervention-derived credits into a lightweight utility estimator. At inference time, semantic relevance is used only for high-recall candidate generation, while predicted utility arbitrates which memories are retained under a fixed token budget, without modifying the underlying agent policy or requiring additional environment interaction. We evaluate MemCredit on ALFWorld and WebShop against representative memory baselines. On ALFWorld, MemCredit achieves 86.57 SR@50 under a 500-token historical-memory budget, compared with 65.67% for BM25 retrieval. Counterfactual analysis further shows that memory utility is sparse, strongly state dependent, and poorly captured by semantic relevance. These results suggest that decision utility provides a more effective principle than relevance alone for long-horizon agent memory management.