Augmenting LLMs with Long-term Memory (LongMem)
Free while signed in. Answers cite the passages they came from.
First page
The curator’s take
Key pointsEnables LLMs to memorize long history via memory-augmented adaptation.
01
Memory-augmented training: Dedicated adaptation training teaches the LLM to retrieve and use its memory of long past context.
02
ICL over long history: Enables in-context learning that spans far longer contexts than the model's raw attention window.
03
Decoupled retrieval: Separates the retrieval mechanism from the main model, allowing memory to grow without increasing model size.
04
Long-context direction: Part of 2023's multi-pronged attack on context-window limits, complementary to position interpolation and ring attention.
Every Monday
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack