LLaSM (Large Language and Speech Model)
Free while signed in. Answers cite the passages they came from.
A combined language-and-speech model trained with cross-modal conversational abilities.
Cross-modal conversation: Supports speech-and-language instructions seamlessly, enabling more natural interactions than text-only or speech-only systems.
Instruction-tuned: Fine-tuned on speech-language instruction data, letting users speak prompts and receive responses without a separate ASR step.
Unified architecture: Uses a single model trained end-to-end rather than a cascade of ASR, LLM, and TTS - reducing error propagation and improving latency.
Accessibility implication: Positions the unified speech-language approach as a path toward more accessible AI interfaces, particularly for users who prefer voice interaction.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack