Audiobox
Meta's Audiobox is a unified flow-matching audio model that generates speech, sound effects, and music from natural-language and example prompts.
Ask this paper
Unified audio generation: Single model handles speech, sound, and music - ending the typical pattern of one model per audio modality.
Description + example prompting: Supports both natural-language descriptions and reference-audio examples for style control, letting users mix semantic and acoustic conditioning.
Self-supervised infilling: Adapts a self-supervised infilling objective to pretrain on large unlabeled audio, reducing dependence on scarce labeled speech/music datasets.
Novel voice/styles: Unlocks generation of novel vocal and acoustic styles by interpolating in the learned audio space, going beyond reproduction of training-set styles.