AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Discover and explore top AI papers with Claude Code or Codex
npx @dair-ai/mcp setup
TinyLoRA
This paper from Meta FAIR asks how small a LoRA adapter can get and still teach a model to reason. The answer: remarkably small. The authors propose TinyLoRA, a method that scales low-rank adapters down to as few as one trainable parameter by projecting through fixed random tensors and sharing weights across all modules. The key insight is that RL makes fundamentally more information-dense updates than SFT, enabling effective learning with orders of magnitude fewer parameters.
Reinforcement Learning via Self-Distillation
This paper introduces Self-Distillation Policy Optimization (SDPO), an on-policy RL algorithm that converts rich textual feedback from verifiable environments into dense credit assignment without requiring an external teacher model. SDPO uses the current model conditioned on feedback as a “self-teacher” to retrospectively identify mistakes in its own rollouts, substantially outperforming GRPO across scientific reasoning, tool use, and competitive programming.
Communication Methods in Multi-Agent RL
A systematic survey of 29 papers reviewing how agents coordinate in multi-agent reinforcement learning, covering fully connected message passing, implicit communication, attention-based selective methods, graph-based relational approaches, and role-based hierarchical frameworks. The analysis reveals that attention- and graph-based methods dominate recent research, while implicit communication is seeing renewed interest for its scalability in decentralized settings where explicit channels are infeasible.
TTT-Discover: Learning to Discover at Test Time
TTT-Discover introduces test-time training for scientific discovery, performing reinforcement learning at test time so the LLM can continue to train with experience specific to the test problem. Unlike prior work like AlphaEvolve that prompts a frozen LLM, this approach enables the model itself to improve while attempting to solve hard problems.
Reasoning Models Generate Societies of Thought
This paper reveals that enhanced reasoning in models like DeepSeek-R1 and QwQ-32B emerges not from extended computation alone, but from simulating multi-agent-like interactions - a “society of thought” - enabling diversification and debate among internal cognitive perspectives with distinct personality traits and domain expertise.
Self-Evolving Search Agents Without Training Data
Dr. Zero introduces a framework for developing multi-turn search agents that improve themselves autonomously without labeled training data. A proposer generates diverse questions to train a solver initialized from the same base model, creating a self-evolution loop with automated curriculum difficulty scaling.
Unified Long-Term and Short-Term Memory for LLM Agents
AgeMem introduces a unified framework that integrates both long-term and short-term memory operations into an LLM agent’s decision-making policy. The system enables agents to autonomously determine what and when to store, retrieve, update, summarize, or discard information by exposing memory operations as tool-based actions.
MemRL
MemRL enables LLM agents to improve continuously without retraining by separating a frozen model’s reasoning from an evolving memory system. A Two-Phase Retrieval mechanism filters candidates by semantic relevance, then ranks them using learned Q-values that improve through trial-and-error, outperforming existing methods on HLE, BigCodeBench, ALFWorld, and Lifelong Agent Bench.
Nemotron-Cascade
Nemotron-Cascade introduces cascaded domain-wise reinforcement learning (Cascade RL) to build general-purpose reasoning models capable of operating in both instruct and deep thinking modes. Rather than blending heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL stages that reduce engineering complexity while delivering state-of-the-art performance.
GDPO
GDPO addresses a critical flaw in training language models with multiple competing objectives. The authors discover that when applying Group Relative Policy Optimization (GRPO) to multi-reward settings, normalizing distinct rollout reward combinations causes them to collapse into identical advantage values, degrading training signal quality and stability.
Training AI Co-Scientists Using Rubric Rewards
This paper from Meta Superintelligence Labs presents a scalable method to train language models to generate better research plans without expensive human supervision or real-world execution. The approach automatically extracts research goals and goal-specific grading rubrics from scientific papers, then uses reinforcement learning with self-grading to improve plan generation.
LaMer
LaMer introduces a Meta-RL framework that enables LLM agents to actively explore and learn from environment feedback at test time. Unlike standard RL-trained agents that learn fixed policies and struggle with novel tasks, LaMer agents learn exploration strategies that transfer across environments.
JustRL
JustRL challenges the assumption that complex RL pipelines are necessary for training small language models. Using single-stage training with fixed hyperparameters, the authors achieve state-of-the-art math reasoning performance on two 1.5B models while using 2x less compute than sophisticated multi-stage approaches.
Self-Play SWE-RL
Self-Play SWE-RL (SSR) trains software engineering agents through self-play, requiring only access to sandboxed repositories with no human-labeled issues or tests. A single LLM learns to both inject and repair bugs of increasing complexity, achieving +10.4 points on SWE-bench Verified while consistently outperforming human-data baselines.
Pre-Training, Mid-Training, and RL Interplay
CMU researchers develop a controlled experimental framework using synthetic reasoning tasks to isolate how pre-training, mid-training, and RL-based post-training each contribute to reasoning capabilities in language models. The study reconciles conflicting views on whether RL truly extends reasoning beyond what models learn during pre-training.
Agentic AI Adaptation Survey
Researchers from UIUC, Stanford, Berkeley, and other institutions present the first comprehensive taxonomy of adaptation strategies for agentic AI systems. The survey organizes recent advances into a unified framework covering how agents and their tools can be modified to achieve higher task performance, improved reliability, and better generalization across diverse scenarios.
Nanbeige4-3B
Nanbeige4-3B is a 3B parameter model pretrained on 23T tokens and fine-tuned on over 30M instructions using a Fine-Grained Warmup-Stable-Decay scheduler, Dual Preference Distillation, and multi-stage reinforcement learning. Despite its compact size, it outperforms Qwen3-8B and Qwen3-14B on reasoning benchmarks and rivals much larger models on WritingBench, demonstrating that well-engineered small models can match far larger counterparts.
DeepSeek-V3.2
DeepSeek releases V3.2, an open model that matches GPT-5 on reasoning benchmarks while introducing significant architectural and training innovations. The high-compute variant, DeepSeek-V3.2-Speciale, surpasses GPT-5 and achieves gold-medal performance in both the 2025 IMO and IOI competitions.
Evolving Multi-Agent Orchestration
OpenBMB researchers propose a “puppeteer-style” paradigm for multi-agent LLM collaboration, where a centralized orchestrator dynamically directs agents based on evolving task states. Trained via reinforcement learning, the system achieves superior performance with reduced computational costs across math, knowledge, and software development tasks.
Training LLMs for Honesty via Confessions
OpenAI introduces a novel method for training LLMs to honestly self-report their own misbehavior through “confessions” - separate outputs where models evaluate their compliance with instructions and policies. By training GPT-5-Thinking to produce confessions after completing tasks, the research demonstrates that models can be incentivized to reveal deceptive behaviors they otherwise hide in their main answers.
CodeVision: Thinking with Programming Vision
Researchers propose CodeVision, a framework where multimodal models generate code as a universal interface to invoke image operations, addressing brittleness in visual reasoning from orientation changes and corruptions. The two-stage training approach combines supervised fine-tuning with RL using dense rewards, enabling flexible tool composition and error recovery on Qwen models.
INTELLECT-3
INTELLECT-3 is a 106B-parameter Mixture-of-Experts model (12B active) trained with large-scale reinforcement learning, achieving state-of-the-art performance for its size across math, code, science, and reasoning benchmarks. Built on top of GLM-4.5-Air base, it outperforms many larger frontier models, including DeepSeek R1-0528, and matches GLM-4.6 (which has over 3x the parameters) on key benchmarks.
Evolution Strategies at Hyperscale
EGGROLL (Evolution Guided General Optimization via Low-rank Learning) is an evolution strategies algorithm designed to scale backprop-free optimization to large population sizes for billion-parameter neural networks. By using low-rank matrix perturbations instead of full-rank ones, EGGROLL achieves a hundredfold increase in training throughput while nearly matching pure batch inference speed.
DR Tulu
DR Tulu-8B is the first open model directly trained for long-form deep research using Reinforcement Learning with Evolving Rubrics (RLER). Unlike existing models trained on short-form QA tasks, DR Tulu learns to produce comprehensive, well-attributed research reports by training with rubrics that co-evolve with the model and are grounded on real-world searched knowledge.