Multimodal · Training

Generative Pretraining in Multimodality (Emu)

First page

Paper summary

A transformer-based multimodal foundation model for generating images and text.

Ask this paper

Key points

Unified pretraining: Pretrains on mixed image-text sequences to generate either modality in multimodal context.

Instruction tuning for assistants: Combines generative pretraining with instruction tuning to produce performant multimodal assistants.

In-context multimodal: Supports in-context learning across images and text, enabling few-shot multimodal tasks.

Multi-modal assistants: Part of the 2023 push (alongside LLaVA, MiniGPT-4) that established the pattern of visual-instruction-tuned assistants.

Every Monday