A Survey on Latent Reasoning
Provides a comprehensive overview of latent reasoning, an emerging field that shifts AI reasoning from explicit, token-based "chain-of-thought" to implicit computations within a model's continuous hidden states. Key ideas:
Ask this paper
Beyond Explicit Reasoning – While traditional Chain-of-Thought (CoT) improves transparency, it is limited by the constraints of natural language. Latent reasoning overcomes this by performing multi-step inference directly in the model's hidden state, unlocking more expressive and efficient reasoning pathways.
Two Paths to Deeper Thinking – The survey identifies two main approaches to latent reasoning: vertical recurrence, where models loop through the same layers to refine their understanding, and horizontal recurrence, where models evolve a compressed hidden state over long sequences of information. Both methods aim to increase computational depth without altering the model's core architecture.
The Rise of Infinite-Depth Models – The paper explores advanced paradigms like text diffusion models, which enable infinite-depth reasoning. These models can iteratively refine an entire sequence of thought in parallel, allowing for global planning and self-correction, a significant leap beyond the fixed, sequential nature of traditional autoregressive models.