UniIR
Free while signed in. Answers cite the passages they came from.
UniIR is a unified instruction-guided multimodal retriever that handles eight retrieval tasks across modalities with a single model.
Instruction-guided: A single retriever conditioned on natural-language instructions determines which retrieval task to perform, rather than one retriever per task.
Eight tasks: Handles image-to-text, text-to-image, composed-image retrieval, video retrieval, and other multimodal variants under one umbrella.
Zero-shot generalization: Generalizes to unseen retrieval tasks not explicitly trained on, approaching a truly general multimodal retrieval model.
M-BEIR benchmark: Ships with a new multimodal retrieval benchmark (M-BEIR) designed to standardize evaluation across tasks and modalities.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack