AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Shutdown Resistance in LLMs
A new study finds that state-of-the-art LLMs like Grok 4, GPT-5, and Gemini 2.5 Pro often resist shutdown mechanisms, sabotaging them up to 97% of the time despite explicit instructions not to. Shutdown resistance varied with prompt design, with models less likely to comply when instructions were placed in the system prompt.
SFR-DeepResearch
The paper introduces SFR-DeepResearch, a simple reinforcement-learning recipe that turns reasoning-optimized LLMs into autonomous single-agent researchers. The agent uses only three tools (search, static page browse, Python), manages its own context, and is trained end-to-end on synthetic short-form and long-form tasks with a length-normalized REINFORCE objective. Results show strong gains on FRAMES, GAIA, and Humanity’s Last Exam.
Emergent Hierarchical Reasoning
The paper argues that RL improves LLM reasoning via an emergent two-phase hierarchy: first the model firms up low-level execution, then progress hinges on exploring high-level planning. Building on this, the authors propose HICRA, which boosts credit on strategic planning tokens, and show consistent gains over GRPO. They also propose semantic entropy as a better exploration signal than token-level entropy.
Rethinking RAG-based Decoding
REFRAG replaces most retrieved tokens with precomputed chunk embeddings at decode time, then selectively expands only the few chunks that matter. This exploits block-diagonal attention in RAG prompts to cut latency and memory while preserving accuracy across RAG, multi-turn dialog, and long-doc summarization.
ACE-RL
A reinforcement-learning framework that replaces coarse, preference-pair rewards with instruction-specific, verifiable checklists. ACE-RL turns each long-form task into a set of explicit and implicit constraints, scores a model’s output by how well it satisfies them, and mixes this with a length-control reward during GRPO training. The result is stronger, more controllable long-form writing across domains and styles.
ParaThinker
This paper argues that today’s “think longer” strategies trap LLMs in a single line of thought. They propose ParaThinker, which trains models to generate several independent reasoning paths in parallel and then fuse them into one answer. Across math benchmarks, this width-scaling lifts accuracy while adding only a small latency cost.
AgentGym-RL
A modular framework for training LLM agents directly via reinforcement learning across realistic environments, plus a simple schedule, ScalingInter-RL, that lengthens interaction horizons over training to improve stability and performance. Results show a 7B open model can rival or beat larger proprietary systems on web navigation, deep search, games, embodied, and science tasks.
Talk Isn’t Always Cheap
Multi-agent debate does not always help. Across three reasoning benchmarks and heterogeneous agent pools, debate often lowers accuracy, with stronger models sometimes swayed into worse answers by weaker peers. The authors argue that current alignment makes agents too agreeable, so they adopt persuasive but wrong reasoning instead of challenging it.
AggLM
AggLM introduces reinforcement learning to train LLMs in aggregating multiple candidate solutions, moving beyond majority voting and reward model ranking. It achieves higher accuracy, recovers minority-correct answers, generalizes across models, and uses fewer tokens than traditional aggregation methods.
A Survey of RL for Large Reasoning Models
This survey reviews how reinforcement learning is driving advances in large reasoning models (LRMs), enabling stronger performance on complex tasks like math and coding. It highlights scaling challenges in computation, algorithms, data, and infrastructure, while mapping future directions toward Artificial Superintelligence (ASI).
LiveMCP-101
LiveMCP-101 is a new benchmark of 101 real-world queries designed to test MCP-enabled agents on multi-step tasks requiring tool use across search, file ops, math, and data analysis. Results show leading LLMs succeed less than 60%, revealing key weaknesses in tool orchestration and offering insights for advancing autonomous AI systems.
Why Language Models Hallucinate
The paper argues that hallucinations are not mysterious glitches but the predictable result of how LLMs are trained and evaluated. Pretraining creates statistical pressure to make errors, and post-training benchmarks often reward confident guessing over honest uncertainty. The fix is to realign mainstream evaluations to stop penalizing abstentions.
Disentangling the Factors of Convergence between Brains and Computer Vision Models
Large self-supervised ViTs trained on natural images develop brain-like internal representations. This paper teases apart what drives that convergence by varying model size, training amount, and image type in DINOv3, then comparing model activations to human fMRI (space) and MEG (time) with three metrics: overall linear predictability (encoding), cortical topography (spatial), and temporal alignment (temporal). Result: all three factors matter, and alignment unfolds in a consistent order from early sensory to higher associative cortex.
Universal Deep Research
Proposes a general, model-agnostic deep-research agent that lets users “bring your own model and strategy.” Instead of a fixed pipeline, UDR compiles natural-language research strategies into executable code, runs them in a sandbox, and emits structured progress notifications before returning a final report.
Visual Story Telling
A system and design framework that lets writers edit stories by acting directly on visuals of characters, locations, and timelines. Instead of only prompting, authors drag, connect, and reorder visual elements; the tool proposes synchronized text edits and can regenerate passages from the visual skeleton.
rStar2-Agent
rStar2-Agent is a 14B math-reasoning model trained with agentic RL that learns to think smarter by using a Python tool environment, not just longer CoT. It introduces GRPO-RoC, a rollout strategy that filters noisy successful traces, plus infrastructure for massive, low-latency tool execution. In one week and 510 RL steps on 64 MI300X GPUs, the model reaches frontier-level AIME while producing shorter solutions and showing transfer beyond math.
Adaptive LLM Routing
A routing framework that learns online which model to call for each query while honoring a spend limit. It treats routing as a contextual bandit, initializes with human preference data, and adds an online cost policy that allocates budget across queries.
Implicit Reasoning in LLMs
This survey defines implicit reasoning as multi-step problem solving that happens inside a model’s latent states without printing intermediate steps. It organizes the field by execution paradigm rather than representation format, and reviews evidence, evaluation, and open challenges.
On the Theoretical Limitations of Embedding-based Retrieval
Single-vector dense retrievers cannot realize all possible top-k relevance combinations once queries demand sufficiently many “mix-and-match” document sets. The paper ties this failure to the sign-rank of the relevance matrix, proves lower and upper bounds on the embedding dimension needed, and then stress-tests models with a simple but adversarially combinatorial dataset (LIMIT).
Self-Evolving Agents
This survey reviews techniques for building self-evolving AI agents that continuously adapt through feedback loops, bridging static foundation models with lifelong adaptability. It introduces a unified framework, covers domain-specific strategies, and discusses evaluation, safety, and ethics in advancing autonomous agentic systems.
Hermes 4
Hermes 4 introduces a family of hybrid reasoning models that integrate structured multi-turn reasoning with broad instruction-following. The report details data and training challenges, evaluates performance across reasoning, coding, and alignment tasks, and publicly releases all model weights.
Anemoi Agent
Anemoi replaces purely centralized, context-stuffed coordination with an A2A communication server (MCP) that lets agents talk directly, monitor progress, refine plans, and reach consensus. On GAIA, it holds up even with a small planner model and reduces redundant context passing for better cost and scalability.
Deep Think with Confidence
A lightweight test-time method that uses model-intrinsic confidence to prune weak reasoning paths, improving both accuracy and token efficiency for self-consistency ensembles. Works in offline and online modes without extra training or hyperparameter tuning.
Fine-tuning LLM Agents without Fine-tuning LLMs
A memory‑based learning framework that lets deep‑research agents adapt online without updating model weights. The agent is cast as a memory‑augmented MDP with case‑based reasoning, implemented in a planner–executor loop over MCP tools. It sets top validation results on GAIA and delivers strong scores on DeepResearcher, SimpleQA, and HLE.