AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
PivotRL
PivotRL is a turn-level reinforcement learning algorithm from NVIDIA designed to tractably post-train large language models for long-horizon agentic tasks. The method operates on existing SFT trajectories, combining the compute efficiency of supervised fine-tuning with the out-of-domain accuracy of end-to-end RL. PivotRL identifies “pivots,” informative intermediate turns where sampled actions exhibit high variance in outcomes, and focuses training signal on these critical decision points. The approach achieves +4.17% higher in-domain accuracy and +10.04% higher out-of-domain accuracy compared to standard SFT, while matching end-to-end RL accuracy with 4x fewer rollout turns. PivotRL is adopted by NVIDIA’s Nemotron-3-Super-120B-A12B as the workhorse for production-scale agentic post-training.
KARL
Databricks presents KARL, a system for training enterprise search agents via reinforcement learning that achieves state-of-the-art performance across a diverse suite of hard-to-verify agentic search tasks. The work also introduces KARLBench, a new evaluation framework spanning six search domains.
Bayesian Teaching for LLMs
Google researchers introduce a method to teach LLMs to reason like Bayesians by fine-tuning on interactions with a Bayesian Assistant that represents optimal probabilistic inference. LLMs normally fall far short of normative Bayesian reasoning, but this training approach dramatically improves their ability to update predictions based on new evidence.
PAHF
Meta introduces PAHF (Personalized Agents from Human Feedback), a continual agent personalization framework that addresses a critical gap: most AI agents cannot adapt to individual user preferences that evolve over time. PAHF couples explicit per-user memory with both proactive and reactive feedback mechanisms.
Learning to Rewrite Tool Descriptions
Intuit AI Research addresses a bottleneck in LLM-agent tool use: tool descriptions are written for humans, not agents. They introduce Trace-Free+, a curriculum learning framework that optimizes tool descriptions without relying on execution traces. The approach delivers consistent gains on unseen tools, strong cross-domain generalization, and robustness as the number of candidate tools scales to over 100, demonstrating that improving tool interfaces is a practical complement to agent fine-tuning.
SkillRL
SkillRL introduces a recursive skill-augmented RL framework that bridges the gap between raw experience and policy improvement through automatic skill discovery. Instead of storing noisy raw trajectories, SkillRL distills experience into reusable high-level behavioral patterns and evolves them alongside the agent policy during training.
EchoJEPA
EchoJEPA is a latent predictive foundation model for echocardiography trained on 18 million echocardiograms from 300,000 patients. By learning to predict in latent space rather than pixel space, the model separates clinically meaningful anatomical signals from ultrasound noise and artifacts, producing representations that dramatically outperform existing approaches on cardiac assessment tasks.
AgentArk
AgentArk distills multi-agent debate dynamics into a single LLM, transferring the reasoning and self-correction abilities of multi-agent systems into one model at training time. Three hierarchical distillation strategies (reasoning-enhanced SFT, trajectory-based augmentation, and process-aware distillation with a process reward model) yield an average 4.8% improvement over single-agent baselines across math and reasoning benchmarks, approaching full multi-agent performance at a fraction of the inference cost. Cross-family distillation (e.g., Qwen3-32B to LLaMA-3-8B) produces the largest gains, suggesting heterogeneous architectures benefit most from transferred reasoning signals.
AgentSkiller
AgentSkiller scales generalist agent intelligence through semantically integrated cross-domain data synthesis, producing 11K high-quality synthetic trajectories across diverse tool-use scenarios. The resulting 14B model beats GPT-o3 on tau2-bench (79.1% vs 68.4%), and even the 4B variant outperforms 70B and 235B models, demonstrating that data quality and semantic integration matter more than parameter count for building strong tool-use agents.
TinyLoRA
This paper from Meta FAIR asks how small a LoRA adapter can get and still teach a model to reason. The answer: remarkably small. The authors propose TinyLoRA, a method that scales low-rank adapters down to as few as one trainable parameter by projecting through fixed random tensors and sharing weights across all modules. The key insight is that RL makes fundamentally more information-dense updates than SFT, enabling effective learning with orders of magnitude fewer parameters.
InfMem
InfMem is a cognitive agent for ultra-long document QA that uses System-2-style control to actively manage bounded memory. Instead of passively compressing each chunk as it streams in, InfMem runs a PreThink-Retrieve-Write loop that monitors evidence sufficiency, fetches missing facts from anywhere in the document, and compresses everything into a fixed-size memory - then stops early once it has enough.
Kimi K2.5: Visual Agentic Intelligence
Kimi K2.5 is an open-source multimodal agentic model from Moonshot AI that jointly optimizes text and vision capabilities through native multimodal pretraining on 15 trillion mixed tokens, zero-vision SFT, and joint reinforcement learning. K2.5 also introduces Agent Swarm, a parallel agent orchestration framework that dynamically decomposes complex tasks into concurrent subtasks, reducing latency by up to 4.5x over single-agent baselines.
Shaping Capabilities with Token-Level Data Filtering
Researchers from Anthropic and Stanford show that filtering pretraining data at the token level is a highly effective, scalable, and robust approach for selectively removing undesired capabilities from language models. Using medical knowledge removal as a proxy task, token-level filtering Pareto dominates document-level filtering and achieves a 7,000x compute slowdown on the target domain for 1.8B parameter models - while preserving capabilities in related fields.
Reinforcement Learning via Self-Distillation
This paper introduces Self-Distillation Policy Optimization (SDPO), an on-policy RL algorithm that converts rich textual feedback from verifiable environments into dense credit assignment without requiring an external teacher model. SDPO uses the current model conditioned on feedback as a “self-teacher” to retrospectively identify mistakes in its own rollouts, substantially outperforming GRPO across scientific reasoning, tool use, and competitive programming.
Self-Improving Pretraining
Self-Improving Pretraining is a new pretraining paradigm from Meta FAIR that replaces standard next-token prediction with sequence-level generation guided by an existing post-trained model acting as both a suffix rewriter and a suffix judge. The approach addresses quality, safety, and factuality issues at pretraining time rather than deferring them to post-training, yielding large gains across all three dimensions.
LingBot-World: Open-Source World Simulator
LingBot-World is an open-source world simulator that evolves a video generation model into an interactive, real-time environment engine. Built on a 28B-parameter Mixture-of-Experts architecture, it achieves high-fidelity dynamics across diverse domains with sub-second latency at 16 fps, outperforming Genie 3 and Mirage 2 in dynamic degree while being fully open-source.
TTT-Discover: Learning to Discover at Test Time
TTT-Discover introduces test-time training for scientific discovery, performing reinforcement learning at test time so the LLM can continue to train with experience specific to the test problem. Unlike prior work like AlphaEvolve that prompts a frozen LLM, this approach enables the model itself to improve while attempting to solve hard problems.
Reasoning Models Generate Societies of Thought
This paper reveals that enhanced reasoning in models like DeepSeek-R1 and QwQ-32B emerges not from extended computation alone, but from simulating multi-agent-like interactions - a “society of thought” - enabling diversification and debate among internal cognitive perspectives with distinct personality traits and domain expertise.
Learning Latent Action World Models In The Wild
Meta AI researchers address learning world models from in-the-wild videos without requiring explicit action labels, expanding beyond simple robotics simulations and video games to real-world video data with diverse embodiments and uncontrolled conditions.
Extending Context by Dropping Positional Embeddings
DroPE introduces a method for extending a language model’s context window after pretraining without expensive long-context fine-tuning. The approach involves removing positional embeddings from a pretrained model and performing brief recalibration at the original context length.
Ministral 3
Mistral AI releases Ministral 3, a family of compact language models (3B, 8B, 14B parameters) designed for compute and memory-constrained applications from mobile to edge deployments. Created through Cascade Distillation (iterative pruning with continued training), each size offers pretrained, instruction-finetuned, and reasoning variants with integrated image understanding, released under Apache 2.0.
On the Slow Death of Scaling
This essay by Sara Hooker challenges the decade-long assumption that scaling compute always leads to better AI performance. It argues that the relationship between training compute and performance is highly uncertain and rapidly changing, with smaller models now routinely outperforming much larger ones.
Nemotron-Cascade
Nemotron-Cascade introduces cascaded domain-wise reinforcement learning (Cascade RL) to build general-purpose reasoning models capable of operating in both instruct and deep thinking modes. Rather than blending heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL stages that reduce engineering complexity while delivering state-of-the-art performance.
GDPO
GDPO addresses a critical flaw in training language models with multiple competing objectives. The authors discover that when applying Group Relative Policy Optimization (GRPO) to multi-reward settings, normalizing distinct rollout reward combinations causes them to collapse into identical advantage values, degrading training signal quality and stability.