System 2 Attention (S2A)
Meta's S2A uses the LLM's own reasoning to decide what context actually matters, regenerating a clean prompt before the final response step.
Advancing Long-Context LLMs
A survey of methodologies for improving Transformer long-context capability across pretraining, fine-tuning, and inference stages.
Parallel Speculative Sampling
Amazon researchers propose a parallel variant of speculative sampling that achieves significant LLM inference speedups with minimal extra parameters.
Mirasol3B
Google's Mirasol3B is a multimodal model that decouples modalities into focused autoregressive components rather than forcing a single fused stream.
Teaching Small LMs to Reason
An approach that teaches smaller language models to explicitly select among reasoning techniques for each problem.
GPQA
A graduate-level Google-proof QA benchmark designed to stress-test reasoning in systems that might exceed human expertise.
Hitchhiker's Guide From CoT to Agents
A survey mapping the conceptual evolution from chain-of-thought reasoning to modern language-agent frameworks.
GAIA
Meta's GAIA is a benchmark for general AI assistants that requires reasoning, multimodal handling, web browsing, and tool use to solve real-world questions.
MedAgents
A collaborative multi-round framework for medical reasoning that uses role-playing LLM agents to improve accuracy and reasoning depth.
TÜLU 2
Allen AI's TÜLU 2 is a suite of improved open instruction-tuned LLMs and an accompanying study of adaptation best practices.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack