Training Agents Inside of Scalable World Models
A scalable imagination-RL recipe that learns a fast, accurate Minecraft simulator and trains a controllable agent entirely offline. The world model supports real-time interactive rollouts on a single GPU and enables the first purely offline “get diamonds” result from raw pixels and low-level mouse and keyboard.
DeepSeek-V3.2-Exp
DeepSeek adds a fine-grained sparse attention mechanism (DeepSeek Sparse Attention, DSA) to the V3.1 “Terminus” backbone and shows large cost reductions on 128K context without notable quality loss. Model and inference code are released.
The Era of Real-World Human Interaction
This work presents a post-training recipe that learns directly from real user conversations instead of static annotator labels. RLHI combines user-guided rewrites (using follow-ups as corrections) with persona-based rewards (ranking sampled candidates via a persona-conditioned reward model). Trained on WildChat conversations, it shows strong improvements in personalization, instruction following, and even transfers to reasoning tasks.
Rethinking JEPA
Apple proposes SALT (Static-teacher Asymmetric Latent Training), a simple 2-stage V-JEPA alternative that first trains a teacher with pixel reconstruction, then freezes it and trains a student to predict the teacher’s latents on masked regions. It removes EMA, decouples teacher and student, and gives a cleaner model selection while being more compute-efficient.
Agent S3
The paper introduces Behavior Best-of-N (bBoN): run many full CUAs in parallel, convert each rollout into a compact behavior narrative, then do comparative selection to pick the best trajectory. With a stronger base agent (Agent S3), this sets the state of the art on OSWorld and generalizes to Windows and Android.
DeepSearch
DeepSearch integrates Monte Carlo Tree Search directly into RL with verifiable rewards, but during training rather than only at inference. The result is broader exploration, better credit assignment, and higher sample efficiency on math reasoning vs strong 1.5B baselines.
Accelerating Diffusion LLMs
A lightweight, learned policy speeds up diffusion-based LLM decoding by deciding which tokens are already “final” and when to stop generation. The authors train a tiny MLP filter on token confidence signals and add an End-of-Text Prediction that halts decoding as soon as [EoT] is reliably produced. On LLaDA-8B-Instruct, this reaches large throughput gains with minimal or no accuracy loss.
Reasoning Traces Tailored for Small Models
Small models often get worse when you SFT them on long, high-quality CoT from big teachers. This paper pinpoints why and fixes it with Reverse Speculative Decoding (RSD): let the teacher propose tokens, but let the student approve them only if they are probable under the student. Result: traces that stay correct while matching the student’s distribution, which small models can actually learn from.
Tool-Use Mixture (TUMIX)
TUMIX is an ensemble recipe for reasoning that mixes text, code execution, and web search, running 15 diverse agents in parallel and passing intermediate answers across rounds. An LLM-judge controls early stopping, giving up to +3.55% accuracy gains over strong tool-augmented baselines on HLE, GPQA-Diamond, and AIME 24/25 while cutting inference cost by ~50%.
PrompCoT 2.0
PromptCoT 2.0 introduces an EM-based loop for synthesizing harder and more diverse reasoning prompts, replacing manual heuristics from PromptCoT 1.0. It enables both self-play and SFT training regimes, achieving new SOTA on reasoning benchmarks like AIME, HMMT, LiveCodeBench, and Codeforces, showing prompt synthesis as a new scaling axis for LLM reasoning.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack