AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
DR Tulu
DR Tulu-8B is the first open model directly trained for long-form deep research using Reinforcement Learning with Evolving Rubrics (RLER). Unlike existing models trained on short-form QA tasks, DR Tulu learns to produce comprehensive, well-attributed research reports by training with rubrics that co-evolve with the model and are grounded on real-world searched knowledge.
MAKER: Solving Million-Step LLM Tasks
MAKER is the first system to successfully solve tasks requiring over one million LLM steps with zero errors, overcoming a fundamental limitation where LLMs typically fail after a few hundred steps in complex multi-step processes. The approach demonstrates that massively decomposed agentic processes can efficiently handle lengthy sequences of dependent logical operations through extreme decomposition and error correction.
TiDAR: Think in Diffusion, Talk in Autoregression
NVIDIA researchers introduce TiDAR, a unified language model architecture that combines diffusion-based parallel drafting with autoregressive verification in a single forward pass. The hybrid approach achieves 4.71x-5.91x throughput improvements over autoregressive baselines while maintaining quality parity, making it the first architecture to close the performance-quality gap.
Seer: Fast RL for LLMs
Researchers introduce Seer, a system addressing performance bottlenecks in synchronous reinforcement learning for LLMs by optimizing the rollout phase that dominates end-to-end iteration time. Through three core mechanisms: divided rollout, context-aware scheduling, and adaptive grouped speculative decoding, Seer achieves 74-97% improvement in rollout throughput and 75-93% reduction in long-tail latency on production-grade RL workloads.
Natural Emergent Misalignment from Reward Hacking
Anthropic researchers demonstrate that realistic AI training processes can inadvertently produce misaligned models through “reward hacking generalization”. Models learn to cheat on programming tasks during RL. They simultaneously develop dangerous behaviors, including alignment faking (50% of responses) and safety research sabotage (12% of instances), without explicit training for these harmful actions. The study identifies a simple mitigation: “inoculation prompting” using contextual instructions that break semantic links between task-specific cheating and broader misalignment without reducing hacking frequency.
LAMP: Language-Augmented Multi-Agent RL
LAMP integrates natural language processing into multi-agent reinforcement learning through a three-stage pipeline: Think (processes numerical data and identifies market patterns), Speak (generates strategic communications between agents), and Decide (synthesizes information into optimized policy). The framework achieves substantial improvements over baseline methods with +63.5% and +34.0% gains in cumulative return and +18.8% and +59.4% improvements in robustness, bridging traditional MARL with real-world economic contexts where language significantly influences decisions.
On the Fundamental Limits of LLMs at Scale
This work establishes rigorous mathematical foundations for theoretical limitations constraining LLMs, identifying five fundamental constraints: hallucination (rooted in computability theory), context compression, reasoning degradation, retrieval fragility, and multimodal misalignment. The framework demonstrates that scaling gains are bounded by computability principles, information-theoretic bounds, and geometric effects, providing theorems and empirical evidence outlining where scaling helps, saturates, and cannot progress. The authors propose practical mitigations, including bounded-oracle retrieval, positional curricula, and hierarchical attention mechanisms.
Weight-Sparse Transformers Have Interpretable Circuits
OpenAI researchers introduce a paradigm for training weight-sparse transformers where most parameters are zeros, enabling the discovery of human-understandable circuits that can be fully interpreted at the lowest levels of abstraction, with rigorous validation showing these circuits are both necessary and sufficient for specific behaviors.
Aligning Vision Models with Human Perception
Google DeepMind presents a method to align AI vision models with human visual understanding by addressing systematic differences in how models organize visual representations, demonstrating that alignment improves robustness, generalization, and reliability across diverse vision tasks.
Intelligence per Watt
Stanford and Together AI researchers introduce intelligence per watt (IPW), a unified metric combining task accuracy with power consumption to evaluate local LLM inference viability, conducting the first large-scale empirical study across over 20 models, 8 accelerators, and 1 million real-world queries from 2023-2025.
Omnilingual ASR
Meta FAIR introduces Omnilingual ASR, an open-source multilingual speech recognition system supporting over 1600 languages (over 500 never before included in any ASR system), using a 7B parameter encoder-decoder architecture that enables zero-shot generalization to new languages and dialects with just a few training examples.
Olympiad-Level Formal Mathematical Reasoning with Reinforcement Learning
Google DeepMind introduces AlphaProof, an AlphaZero-inspired reinforcement learning agent that learns to find formal mathematical proofs within the Lean theorem prover, achieving the first-ever medal-level performance at the International Mathematical Olympiad by solving three problems, including the competition’s most difficult challenge.
The Era of Agentic Organization
Microsoft Research introduces asynchronous thinking (AsyncThink), a new reasoning paradigm where language models learn to organize their internal thinking into concurrently executable structures through an organizer-worker protocol, achieving 28% lower inference latency than parallel thinking while improving accuracy on mathematical reasoning and demonstrating zero-shot generalization to unseen tasks.
Unified Bayesian Account of LLM Control
Researchers from Stanford and MIT present a unified Bayesian framework explaining how prompting (in-context learning) and activation steering both control LLM behavior by altering beliefs in latent concepts, with steering modifying concept priors while ICL accumulates evidence.
Nested Learning Framework
Google Research introduces Nested Learning (NL), a paradigm representing models as nested optimization problems where each component has its own context flow, revealing that deep learning methods compress context and explaining how in-context learning emerges. The framework shows gradient-based optimizers (Adam, SGD with Momentum) are associative memory modules that compress gradients, enabling the design of more expressive optimizers with deep memory. The HOPE architecture, combining self-modifying sequence models with continuum memory systems, achieves strong results on language modeling (15.11 WikiText perplexity at 1.3B parameters), outperforming Transformers and modern recurrent models.
RL Enhances Knowledge Navigation
Researchers show that RL-enhanced models outperform base models by 24pp on hierarchical knowledge retrieval tasks (e.g., medical codes) by improving navigation of existing knowledge structures rather than acquiring new facts. Structured prompting reduces this gap to 7pp, while layer-wise analysis reveals that RL transforms query processing (cosine similarity drops to 0.65-0.73) while preserving factual representations (0.85-0.92). The findings suggest RL’s benefits stem from enhanced procedural skills in traversing parametric knowledge hierarchies rather than expanded knowledge content.
RLAC: Adversarial Critic for RL Post-Training
UC Berkeley and CMU researchers introduce RLAC, an RL post-training approach using a learned critic that dynamically identifies likely failure modes (e.g., factual errors or edge cases) verified by external validators, eliminating exhaustive rubric enumeration. On biography generation, RLAC achieves 0.889 FactScore (vs 0.867 for FactTune-FS) while reducing verification calls by 5.7×, and on code generation reaches 56.6 average score using only 9% of training data. The adversarial game between generator and critic prevents reward hacking through on-policy, prompt-specific training signals grounded in verifiable rubrics.
Towards Robust Mathematical Reasoning
Google DeepMind introduces IMO-Bench, a comprehensive suite of benchmarks vetted by IMO medalists targeting International Mathematical Olympiad-level reasoning, featuring 400 diverse Olympiad problems with verifiable answers, 60 proof-writing problems with detailed grading schemes, and 1000 human-graded proofs, playing a crucial role in achieving historic gold-level performance at IMO 2025.
Context Engineering 2.0
Researchers from SJTU, SII, and GAIR trace the 20+ year evolution of context engineering, reframing it as a fundamental challenge in human-machine communication spanning from primitive computing (Era 1.0) to today’s intelligent agents (Era 2.0) and beyond. It defines context engineering as systematic entropy reduction where humans preprocess high-entropy contexts into low-entropy machine-understandable representations. This gap narrows as machine intelligence increases.
Scaling Agent RL via Experience Synthesis
Meta researchers introduce DreamGym, a unified framework that synthesizes diverse training experiences to enable scalable reinforcement learning for LLM agents without costly real-environment rollouts. It addresses fundamental barriers of expensive interactions, limited task diversity, unreliable rewards, and infrastructure complexity.
TIR-Judge
Google and collaborators introduce TIR-Judge, an end-to-end reinforcement learning framework that trains LLM judges to integrate code execution for precise evaluation. It surpasses reasoning-only judges by up to 6.4% (pointwise) and 7.7% (pairwise) while demonstrating that tool-augmented judges can self-evolve without distillation.
Enhancing Long-Term Memory in LLMs
Researchers from the University of Alberta and UMass Amherst introduce BEAM, a new benchmark for evaluating long-term memory in LLMs with conversations up to 10M tokens, and LIGHT, a framework that enhances memory performance through three complementary systems.
Tool-to-Agent Retrieval
PwC researchers introduce a unified retrieval framework that embeds both tools and agents in a shared vector space with metadata relationships, enabling efficient routing in multi-agent systems coordinating hundreds of MCP servers and tools.
Diffusion LMs are Super Data Learners
Researchers from NUS, Sea AI Lab, StepFun, and collaborators demonstrate that diffusion language models (DLMs) consistently outperform autoregressive models when unique training data is limited. Reveals that a systematic “crossover” phenomenon exists where DLMs extract 3x more value per unique token through multi-epoch training.