Voicebox
Free while signed in. Answers cite the passages they came from.
Meta's all-in-one generative speech model supporting 6 languages and many speech tasks in-context.
Flow-matching training: Uses flow-matching with text-guided context to unify TTS, denoising, editing, and style transfer in one model.
20x faster: Outperforms specialized TTS systems while running 20x faster than prior state-of-the-art diffusion-based speech models.
Speech ICL: Supports in-context learning for speech - give it an audio prompt and it matches the speaker's voice, style, and prosody zero-shot.
Generalist speech: A major step toward generalist speech foundation models that would accelerate with 2024 systems like VoiceCraft and XTTS.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack