Unlimiformer

External datastore: Augments pre-trained encoder-decoder Transformers with a kNN datastore to support arbitrary-length input.

Training-free: No additional training required - works with existing pretrained Transformers.

Long-document tasks: Demonstrates usefulness in long-document summarization where context spans many thousands of tokens.

RAG-enhancer: Could improve the performance of retrieval-enhanced LLMs by providing unlimited lookback over long conversations or documents.