Can LLM Agents Organize Their Own Memory?
Zichuan Fu, Junjia Qi and colleagues at City University of Hong Kong with Tencent Jarvis Lab introduce SoMem, in which an LLM agent organizes its own long-term memory as linked ordinary documents.
Ask this paper
Self-organized memory. The agent decides what to summarize, how to group information into topics and when to revise them. Every summary links back to the original interactions as evidence.
Runtime support. The runtime validates updates and derives temporal, relational and recent-memory views automatically. At query time the agent chooses what to search and follows links to original evidence.
Results. SoMem beats the strongest baseline by 2.66 points on LoCoMo, 3.10 on BEAM-100K and 2.00 on LongMemEval-S.
Reuse. Memory built by one model can be reused by others, and extra reasoning effort at query time further improves answers.
Abstract
In most LLM agents, weights are fixed after deployment, so new information must be retained in external memory. Existing approaches often rely on predefined records and fixed organizational rules, which cannot readily accommodate evolving tasks and interactions. This raises the question: can LLM agents organize their own memory using ordinary documents? We introduce self-organized memory (SoMem), a framework that stores memory in linked documents. The agent decides what to summarize, how to group related information, and when to revise existing topics, and every summary it produces stays linked to the original interactions as evidence. The runtime validates updates and automatically derives temporal, relational, and recent-memory views from the organized content. At query time, the agent chooses what to search and read, following document links and inspecting original evidence as needed. On LoCoMo, BEAM-100K, and LongMemEval-S, SoMem outperforms the strongest baseline by 2.66, 3.10, and 2.00 percentage points, respectively. The constructed memory can be reused across models, and additional reasoning effort can further improve answer quality. These findings demonstrate the feasibility of agents organizing and using their own persistent memory through ordinary documents. Code is available at: https://github.com/Qi202/SoMem.