Memory-R1
Free while signed in. Answers cite the passages they came from.
A framework that teaches LLM agents to decide what to remember and how to use it. Two RL-fine-tuned components work together: a Memory Manager that learns CRUD-style operations on an external store and an Answer Agent that filters retrieved memories via “memory distillation” before answering. Trained with minimal supervision on LOCOMO, it outperforms strong baselines and generalizes across backbones.
Active memory control with RL: The Memory Manager selects ADD, UPDATE, DELETE, or NOOP after a RAG step and edits entries accordingly; training with PPO or GRPO uses downstream QA correctness as the reward, removing the need for per-edit labels.
Selective use of long histories: The Answer Agent retrieves up to 60 candidates, performs memory distillation to keep only what matters, then generates the answer; RL fine-tuning improves answer quality beyond static retrieval.
Data-efficient training: Using only 152 QA pairs for training and outcome-driven rewards, Memory-R1 attains large gains on LOCOMO, highlighting that effective memory behavior can be learned with minimal supervision.
State-of-the-art results: Across LLaMA-3.1-8B and Qwen-2.5-7B backbones, GRPO variants achieve the best overall F1, BLEU-1, and LLM-as-a-Judge scores vs. Mem0, A-Mem, LangMem, and LOCOMO baselines.
Ablations that isolate the wins: RL improves both components individually; memory distillation further boosts the Answer Agent, and gains compound when paired with a stronger memory manager.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack