System 2 Attention (S2A)
Meta's S2A uses the LLM's own reasoning to decide what context actually matters, regenerating a clean prompt before the final response step.
Advancing Long-Context LLMs
A survey of methodologies for improving Transformer long-context capability across pretraining, fine-tuning, and inference stages.
Parallel Speculative Sampling
Amazon researchers propose a parallel variant of speculative sampling that achieves significant LLM inference speedups with minimal extra parameters.
Mirasol3B
Google's Mirasol3B is a multimodal model that decouples modalities into focused autoregressive components rather than forcing a single fused stream.
Teaching Small LMs to Reason
An approach that teaches smaller language models to explicitly select among reasoning techniques for each problem.
GPQA
A graduate-level Google-proof QA benchmark designed to stress-test reasoning in systems that might exceed human expertise.
Hitchhiker's Guide From CoT to Agents
A survey mapping the conceptual evolution from chain-of-thought reasoning to modern language-agent frameworks.
GAIA
Meta's GAIA is a benchmark for general AI assistants that requires reasoning, multimodal handling, web browsing, and tool use to solve real-world questions.
MedAgents
A collaborative multi-round framework for medical reasoning that uses role-playing LLM agents to improve accuracy and reasoning depth.
TÜLU 2
Allen AI's TÜLU 2 is a suite of improved open instruction-tuned LLMs and an accompanying study of adaptation best practices.