Unlimiformer
Free while signed in. Answers cite the passages they came from.
First page
The curator’s take
Key pointsLong-range Transformers with unlimited length input via external datastores.
01
External datastore: Augments pre-trained encoder-decoder Transformers with a kNN datastore to support arbitrary-length input.
02
Training-free: No additional training required - works with existing pretrained Transformers.
03
Long-document tasks: Demonstrates usefulness in long-document summarization where context spans many thousands of tokens.
04
RAG-enhancer: Could improve the performance of retrieval-enhanced LLMs by providing unlimited lookback over long conversations or documents.
Every Monday
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack