AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Discover and explore top AI papers with Claude Code or Codex
npx @dair-ai/mcp setup
A Deep Dive into RL for LLM Reasoning
This paper reviews and rigorously re-evaluates reinforcement learning techniques for LLM reasoning, addressing inconsistencies caused by varied setups and unclear guidelines. It offers a unified open-source framework, practical selection guidelines, and shows that a minimalist two-technique combo with vanilla PPO can outperform methods like GRPO and DAPO.
Is Chain-of-Thought Reasoning a Mirage?
Researchers from Arizona State University investigate whether Chain-of-Thought (CoT) reasoning in LLMs reflects genuine logical inference or mere pattern replication from training data. They introduce a data distribution lens to analyze CoT’s dependence on in-distribution patterns and its brittleness under distribution shifts.
ReaGAN
This paper introduces ReaGAN, a graph learning framework that reconceptualizes each node in a graph as an autonomous agent capable of planning, reasoning, and acting via a frozen LLM. Instead of relying on static, layer-wise message passing, ReaGAN enables node-level autonomy, where each node independently decides whether to aggregate information from local neighbors, retrieve semantically similar but distant nodes, or take no action at all. This node-agent abstraction addresses two key challenges in graph learning: (1) handling varying informativeness of nodes and (2) combining local structure with global semantics.
Tabular Data Understanding with LLMs
This survey reviews LLM and MLLM methods for table understanding, outlining a taxonomy of tabular representations and tasks. It identifies key gaps, including limited reasoning beyond retrieval, difficulties with complex or large-scale tables, and poor generalization across diverse formats.
Medical Reasoning in the Era of LLMs
This review categorizes techniques for enhancing LLM medical reasoning into training-time (e.g., fine-tuning, RL) and test-time (e.g., prompt engineering, multi-agent systems) approaches, applied across modalities and clinical tasks. It highlights advances in evaluation methods, key challenges like the faithfulness–plausibility gap, and the need for native multimodal reasoning in future medical AI.
Geometric-Mean Policy Optimization
Introduces a stabilized alternative to Group Relative Policy Optimization (GRPO), which is widely used to improve reasoning capabilities in large language models via reinforcement learning. GRPO optimizes the arithmetic mean of token-level rewards but suffers from training instability due to extreme importance sampling ratios. GMPO addresses this by instead maximizing the geometric mean of token-level rewards, leading to more stable updates.
Graph-R1
Introduces a novel RAG framework that moves beyond traditional one-shot or chunk-based retrieval by integrating graph-structured knowledge, agentic multi-turn interaction, and RL. The core goal is to improve factual accuracy, retrieval efficiency, and reasoning quality in knowledge-intensive tasks.
Hierarchical Reasoning Model
Hierarchical Reasoning Model (HRM) is a novel, brain-inspired architecture that replaces CoT prompting with a recurrent model designed for deep, latent computation. It departs from token-level reasoning by using two coupled modules: a slow, high-level planner and a fast, low-level executor, achieving greater reasoning depth and efficiency with only 27M parameters and no pretraining. Despite its small size and minimal training data (~1k examples), HRM solves complex tasks like ARC, Sudoku-Extreme, and 30×30 maze navigation, where CoT-based LLMs fail.
Gemini 2.5 Pro Capable of Winning Gold at IMO 2025
This paper demonstrates that Google’s Gemini 2.5 Pro model is capable of solving 5 out of 6 problems from the 2025 International Mathematical Olympiad (IMO), achieving a gold-medal-level performance. The authors construct a rigorous self-verification pipeline to elicit high-quality, fully justified solutions from the model, explicitly addressing concerns around reasoning depth, rigor, and data contamination.
Inverse Scaling in Test-Time Compute
This paper presents a systematic study of inverse scaling in large reasoning models (LRMs), where increasing the test-time compute (i.e., reasoning length) harms rather than helps model performance. The authors introduce benchmark tasks across three categories: simple counting with distractors, regression with spurious features, and deductive logic puzzles, revealing consistent accuracy degradation as reasoning length increases. The work identifies model-specific failure modes and raises critical implications for LLM safety and alignment.
Deep Researcher with Test-Time Diffusion
Rethinks how deep research agents generate long-form reports. Rather than relying on static inference strategies like Chain-of-Thought or best-of-n sampling, TTD-DR frames the report generation process as a diffusion process. It starts with a noisy draft and iteratively refines it through retrieval-enhanced denoising, guided by a structured plan. This iterative loop mimics how human researchers search, reason, revise, and accumulate context over time.
Agentic-R1
This paper introduces Agentic-R1, a 7B language model trained to dynamically switch between tool-based execution and pure text reasoning using a novel fine-tuning framework called DualDistill. Rather than relying solely on long chain-of-thought (long-CoT) reasoning or tool use, the method composes solution trajectories from two specialized teachers, one strong in abstract reasoning (Deepseek-R1) and another in code-based tool use (OpenHands/Claude-3.5). The student learns to select the best strategy per task and improves further via self-distillation.
Chain-of-Thought Monitorability
Proposes that language-based CoT reasoning in LLMs offers an opportunity for AI safety by enabling automated oversight of models’ internal reasoning processes. The authors argue that, while imperfect, CoT monitoring is a promising method for detecting misbehavior, revealing goals, and improving interpretability, but its effectiveness is fragile and must be preserved with care.
Stress Testing Large Reasoning Models
Proposes a new benchmark framework called REST to evaluate the robustness of Large Reasoning Models (LRMs) under multi-question stress. Unlike traditional single-question evaluations, REST tests models by presenting multiple reasoning problems in a single prompt, better simulating real-world multi-tasking demands.
Scaling up RL
This paper investigates how prolonged RL can enhance reasoning abilities in small language models across diverse domains. Building on successes like OpenAI’s O1 and DeepSeek-R1, the authors explore large-scale RL with verifiable rewards and improved policy optimization techniques to enable sustained learning and generalization. They introduce the Nemotron-Research-Reasoning-Qwen-1.5B model and demonstrate substantial gains over baselines using a carefully staged RL training recipe.
A Survey on Latent Reasoning
Provides a comprehensive overview of latent reasoning, an emerging field that shifts AI reasoning from explicit, token-based "chain-of-thought" to implicit computations within a model's continuous hidden states. Key ideas:
Adaptive Branching MCTS
Researchers from Sakana AI introduce Adaptive Branching Monte Carlo Tree Search (AB-MCTS), a new framework that dynamically decides whether to "go wider" (explore new solutions) or "go deeper" (refine existing ones) during inference. Key ideas:
Chain-of-Thought Is Not Explainability
It challenges the common assumption that chain-of-thought (CoT) reasoning in LLMs is synonymous with interpretability. While CoT improves performance and offers a seemingly transparent rationale, the authors argue it is neither necessary nor sufficient for faithful explanation. Through a review of empirical evidence and mechanistic insights, the paper makes the case that CoT often diverges from the internal computations that actually drive model predictions.
NaturalThoughts
This paper introduces NaturalThoughts, a large-scale dataset of reasoning traces distilled from DeepSeek-R1 using questions from the NaturalReasoning corpus. It challenges the "Less is More" hypothesis by showing that simply scaling up high-quality reasoning traces, without aggressive filtering, yields robust and general improvements across STEM reasoning tasks for smaller models like Llama-3.1-8B and Qwen-2.5-7B.
Visual Structures Help Visual Reasoning
This study shows that adding simple spatial structures (like horizontal lines) to images significantly boosts GPT-4o’s visual reasoning by improving feature binding. This visual input tweak outperforms textual strategies alone, yielding large gains in visual search (+25%), counting (+26.8%), and spatial understanding (+9.5%).
MEM1
This work introduces MEM1, an RL framework for training language agents that operate efficiently over long-horizon, multi-turn tasks by learning to consolidate memory and reasoning into a compact internal state. Unlike traditional agents that append all past interactions, leading to ballooning memory usage and degraded performance, MEM1 maintains a constant memory size by discarding obsolete context after each reasoning step. It achieves this by jointly updating an internal state that encodes both new observations and prior memory, optimizing for task completion via RL without needing external memory modules. Key contributions and findings:
DeepRare
Introduces DeepRare, a modular agentic system powered by LLMs to aid rare disease diagnosis from multimodal clinical inputs (text, HPO terms, VCFs). It generates ranked diagnostic hypotheses with fully traceable reasoning chains linked to verifiable medical sources, addressing a long-standing need for interpretability in clinical AI.
Eliciting Reasoning with Cognitive Tools
Proposes a modular, tool-based approach to eliciting reasoning in LLMs, inspired by cognitive science. Rather than relying solely on RL or chain-of-thought prompting, the authors introduce a framework where the LLM calls self-contained "cognitive tools" to modularize and scaffold internal reasoning. These tools encapsulate operations like understanding questions, recalling analogous examples, examining answers, and backtracking. The system is implemented in an agentic tool-calling style, allowing LLMs to dynamically invoke tools during reasoning without extra fine-tuning. Highlights:
SHADE-Arena
This paper introduces SHADE-Arena, the first large-scale benchmark for evaluating whether LLM-based agents can subtly carry out harmful side objectives (sabotage) while avoiding detection by monitors. The dataset contains 17 complex task pairs, each with a benign main task and a related malicious side task, set in realistic, tool-using environments. Agents are evaluated on their ability to complete both tasks without raising suspicion from an LLM monitor. Key Findings: