CHIME: Credit-Aware Hierarchical Memory Evolution for Long-Horizon Agentic Planning
Yongshi Ye, Tian Lan and colleagues (Xiamen University, Alibaba International) propose CHIME, a self-evolving memory framework that fixes the credit assignment problem in experience memory by attributing an outcome before writing it anywhere.
Ask this paper
Attribute before memorize: Existing self-evolving memory writes the final task outcome into one undifferentiated bank, conflating a bad plan with a bad execution or a hostile environment. CHIME first attributes the outcome to the plan, the execution, both or neither.
Two banks, not one: A planning bank and an execution bank are updated separately, which keeps a botched tool call from poisoning the planning experience that led to it.
Fewer, better memory items: CHIME accumulates effective memory with far fewer entries than baselines, and learned memory values track downstream utility, with planning memories worth more than execution memories.
Memory transfers across backbones: The accumulated bank keeps working when the underlying model is swapped, which is the property that makes memory an asset rather than a per-model artifact.
Why it matters: Credit assignment is the unglamorous bottleneck in every self-improving agent, and 'attribute the outcome before you store it' is a cheap, general fix that does not need training.
Abstract
Planning is a central capability that enables agents to decompose complex long-horizon tasks into manageable steps. Test-time search and training-based methods improve planning but incur high inference costs or require expensive training data. Self-evolving memory instead accumulates reusable experience from agent interaction outcomes into an external memory bank, so planning capability keeps improving at inference time without parameter updates. However, existing self-evolving memory methods share an inherent credit assignment problem: they rely on final task outcomes as feedback, but such outcomes conflate plan quality with execution errors and environmental factors, so the accumulated planning experience is often biased and noisy. To address this problem, we propose Credit-Aware Hierarchical Memory Evolution (CHIME), a self-evolving memory framework that maintains a separate planning bank and execution bank and follows an attribute-before-memorize principle: CHIME first attributes each task outcome to the plan, the execution, both, or neither, and then updates only the corresponding memory bank. Extensive experiments on four long-horizon agent benchmarks show that CHIME consistently outperforms state-of-the-art training-based and self-evolving memory baselines. Further analyses reveal several interesting findings. For example, CHIME accumulates effective memory with far fewer items. In addition, the learned memory values faithfully reflect downstream utility: high-quality planning memories are more valuable than execution memories. Finally, the accumulated memory effectively transfers across backbone models. Code will be released at https://github.com/ATH-MaaS/Marco-DeepResearch.