Memex(RL)
Free while signed in. Answers cite the passages they came from.
As tasks get longer and more complex, LLM agents lose track of what they have learned, what they have tried, and what still needs to be done. Memex(RL) introduces an indexed experience memory mechanism that scales agent capability on long-horizon tasks without discarding evidence or blowing up the context window.
Indexed experience memory: Rather than lossy compression, Memex maintains a compact working context consisting of concise structured summaries and stable indices while storing full-fidelity underlying interactions in an external experience database. The agent decides what to summarize, what to archive, how to index it, and when to retrieve it.
RL-optimized memory operations: The MemexRL reinforcement learning framework optimizes both write and read behaviors with reward shaping tailored to indexed memory usage under a context budget. This teaches the agent to manage its own memory strategically rather than relying on fixed heuristics.
Bounded retrieval complexity: Theoretical analysis demonstrates that Memex can maintain decision quality with bounded retrieval operations while keeping computational load manageable as task history grows. This makes the approach practical for tasks that span hundreds or thousands of steps.
Smaller context, better results: Empirically, agents trained with MemexRL improve task success rates on challenging long-horizon tasks while using a significantly smaller working context than baseline approaches. Less context, used more intelligently, outperforms brute-force context expansion.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack