CM3Leon
Free while signed in. Answers cite the passages they came from.
Meta's retrieval-augmented multi-modal language model that generates both text and images.
Autoregressive multi-modal: Unifies text and image generation in a single autoregressive token-based architecture, handling both modalities in any order.
5x training efficiency: Achieves SOTA image generation quality with 5x less training compute than comparable methods due to retrieval augmentation and instruction tuning.
Instruction tuning for images: Demonstrates that supervised fine-tuning and instruction tuning - originally developed for LLMs - also massively improves multimodal generation quality.
Any-to-any direction: Early proof-of-concept for unified any-to-any multi-modal models, pre-dating and inspiring 2024 systems like Chameleon and GPT-4o.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack