RECOMP (Retrieval-Augmented LMs with Compressors)
Free while signed in. Answers cite the passages they came from.
Proposes two compression approaches to shrink retrieved documents before in-context use.
Extractive compressor: Selects the most useful sentences from retrieved documents, retaining the most relevant signal at a fraction of token budget.
Abstractive compressor: Generates a summary synthesizing information from multiple retrieved documents, compressing redundancy across sources.
6% compression rate: Achieves compression rates as low as 6% with minimal performance loss on language modeling and open-domain QA.
Selective augmentation: The training scheme learns to emit empty summaries when retrieved docs are irrelevant - a built-in mechanism for gracefully handling noisy retrieval.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack