AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
RLMF
LLMs routinely hallucinate with high confidence, miss their own knowledge boundaries, and misreport uncertainty, and most fixes bolt calibration on from the outside. RLMF, a Google and Yale collaboration, instead turns the model’s own metacognition into the training signal. ---
Reasoning Quality Emerges Early
Curating reasoning data is expensive because scoring a trace usually means reading it to the end, but this UCLA work shows the quality of a trace is largely decided in its opening tokens. A short prefix predicts whole-trace quality well enough to rank and filter on, and difficulty can be detected from the loss of the first 100 tokens at a perturbed checkpoint. That turns curation into a cheap early-stopping problem, outperforming baselines while being far more token efficient at building SFT data for reasoning models.
Sakana Fugu
Frontier LLMs keep advancing, and different providers are increasingly specializing in distinct domains, which raises a natural next objective: how do you combine those individual specializations into one collectively intelligent system? Sakana Fugu answers with a family of orchestrator models that are themselves language models trained to read a user query and dynamically devise the agentic scaffold needed to solve it.
Autodata
Building synthetic training data has mostly stayed a fixed pipeline that you hand-tune once and then freeze. Autodata rethinks that by casting an AI agent as a data scientist that builds high-quality training and evaluation data, then meta-optimizes that agent so it learns to create even stronger data over time.
Skill-MAS
Automatic generation of multi-agent systems is stuck between inference-time methods that reuse frozen frontier models but never learn, and training-time methods that internalize experience through gradient updates but are capped by the weaker models small enough to fine-tune. Skill-MAS proposes a third path that treats high-level orchestration as an evolvable Meta-Skill, decoupling experience retention from weight updates so frontier models keep getting better at orchestration without any gradient steps. Across four complex benchmarks and four distinct LLMs it delivers strong, transferable gains at a favorable cost-performance trade-off.
SpatialClaw
Spatial reasoning over 3D and 4D scenes is still where general vision-language models break down, because they emit a text answer directly rather than measuring anything. From NVIDIA, SpatialClaw is a training-free framework that rethinks the action interface and lets a VLM-backed agent reason through code instead. The agent writes one Python cell per step into a persistent Jupyter kernel preloaded with perception primitives and scientific libraries, then inspects intermediate results and revises its strategy across steps.
From Trainee to Trainer
Who should design the training environment for an RL agent, the practitioner or the policy itself? RL pipelines for LLMs usually rely on manually redesigned environments between stages, with practitioners guessing which configuration will best improve the current policy. This paper hands that job to the model, proposing an LLM-as-Environment-Engineer framework where the policy diagnoses its own weaknesses and proposes the next environment to train on.
OpenClaw-Skill
Equipping LLM agents with effective skills is most of the battle in real systems, yet most skill-induction work distills one trajectory at a time, which produces narrow, brittle skills. OpenClaw-Skill introduces Collective Skill Tree Search, a tree-search-based skill construction framework that builds a structured, diverse, and generalizable tree of skills, then trains agents to actually use what it builds.
The Stanford EDGAR Filings Dataset
Clean, long-context documents remain scarce for pretraining, especially in finance. This release reconstructs U.S. SEC corporate and financial disclosures into layout-faithful, token-efficient MultiMarkdown, publishing 152B tokens in SEFD-v1 out of an estimated 550B-token archive spanning 18.5M filings, with less than 0.1% overlap with Common Crawl corpora. It also ships two derived benchmarks, EDGAR-Forecast for numerical forecasting and EDGAR-OCR for financial table transcription, to support financial reasoning, forecasting, and document understanding.
Self-Harness
Most agent scaffolds are built once by hand and then frozen, even as the underlying models keep changing. This paper introduces Self-Harness, a paradigm where an LLM agent improves its own operating harness, the prompts, tools, memory, and orchestration around the base model, without human engineers or a stronger external agent. Because every model fails in its own way, the system mines those model-specific weaknesses and turns them into concrete, executable harness edits rather than generic advice.
Agentopia
Agentopia is one of the most ambitious agent-society testbeds yet, a 79-page release that drops 100 LLM agents into a persistent world and lets them live, form relationships, and pursue goals over 10 simulated years, a horizon orders of magnitude longer than prior day-level work. Beyond observing emergent social behavior, the authors use the simulation as a training signal, optimizing models toward a life reward that reflects human well-being via rejection sampling.
The Geometry of On-Policy Distillation
On-policy distillation (OPD) has become one of the most discussed post-training recipes of the year, but it has mostly been treated as a black box sitting somewhere between supervised fine-tuning and RL. This paper opens it up, characterizing how OPD changes a model's weights at the level of parameter geometry, and argues OPD is not a midpoint between SFT and RLVR but its own distinct kind of update.
Beyond Scalar Rewards
Reward models usually compress a judgment into a single scalar, but this paper argues human preferences are better captured as score distributions, and proposes Z-Reward, which internalizes reasoning into a predicted distribution before scoring. A large vision-language teacher does the reasoning-heavy judgment and is distilled into a compact student for efficient deployment, with the 27B teacher reaching 89.6% human-preference accuracy and the 9B student nearly matching it at 88.6%. Used as a reinforcement learning signal, it delivers a 41.3% net preference improvement over a supervised baseline, beating GRPO and other reward methods.
LEAP
New research from Google shows how far a custom agent harness can push a general-purpose model on formal mathematics. LEAP wraps a general LLM in an agentic scaffold that grounds every step in the Lean compiler and iterates against verifier feedback. Rather than fine-tuning a specialized prover, it leans on informal reasoning, instruction following, and self-refinement, then forces every formal step through a compiler check before moving on.
Scaling Laws for Agent Harnesses
Most harness tuning treats every token and tool call as if volume is what counts. This paper shows that it mostly does not, and introduces Effective Feedback Compute (EFC), a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for later decisions, then normalizes by task demand.
Learn From Your Own Latents
LLMs learn by predicting tokens, while world models like JEPA and data2vec learn by predicting their own internal representations. This paper provides a sample-complexity theory for why the second approach can be dramatically more data-efficient, using a tractable probabilistic context-free grammar as the analytical setting where compositional structure can be measured exactly.
A Primer on Post-Training Reasoning Data
This primer is the first to pull the scattered post-training reasoning-data literature into one place, synthesizing over 150 public studies and system reports that previously lived across dataset papers, RL write-ups, and lab reports. It organizes the field around four questions: what data objects exist, what makes them useful, how they are constructed, and how they scale. The key reframing is that a reasoning-data item is more than a prompt-response pair: it packages a problem or state, model behavior, judging feedback, and attribution metadata, with usefulness defined relative to the verifier and the rest of the corpus rather than in isolation.
SkillOpt
Microsoft Research treats a compact natural-language skill document as the trainable state of a frozen agent, then learns that document through rollouts, reflection, and bounded edits gated by held-out validation. The argument is direct: most engineers handwrite agent skill docs and hope they generalize, when the doc itself should be optimized like a parameter. SkillOpt reframes the SKILL.md file as an external parameter of a model whose weights never change.
Compiling Agentic Workflows into Weights
This paper shows that a full agentic workflow can be distilled into the weights of a small model and run at roughly two orders of magnitude lower inference cost while preserving near-frontier task quality. Instead of keeping an external orchestrator above the LLM, the procedure is compiled into the weights of a fine-tuned model, producing what the authors call a subterranean agent.
Adapting the Interface, Not the Model
When a frozen LLM agent repeatedly fails in a deterministic, rule-governed environment, do you have to retrain the model? Life-Harness argues no. Many failures come from mismatches at the model-environment interface, not from the model's reasoning, so the fix belongs in the runtime harness. Life-Harness is a lifecycle-aware harness that improves frozen agents without touching model weights or the evaluation environment.
Epicure
Epicure trains a family of multilingual ingredient embeddings from scratch on 4.14 million recipes aggregated from 11 sources across seven languages, with raw ingredient strings normalized to 1,790 canonical entries via an LLM-augmented pipeline. It ships three skip-gram (Metapath2Vec) variants that share architecture but differ in what they walk: recipe co-occurrence only, chemical-compound structure from FlavorDB only, or a blend of both, placing each model at a different point on the chemistry-versus-recipe-context spectrum. The result is a compact, downloadable map of the emergent geometry of food, a clean reminder that representation learning generalizes well beyond text into surprisingly everyday domains.
Memory as a Model
MeMo augments any frozen LLM with a separately trained memory model that stores, retrieves, and integrates facts on the base model's behalf. Memory updates are decoupled from base-model weight updates, so the system supports continual learning without catastrophic forgetting, a property RAG fails to deliver because a vector store is just a database with a learned encoder bolted on.
Lighthouse Attention
Nous Research proposes a training-only attention wrapper for long-context pretraining. Lighthouse Attention wraps standard SDPA with a hierarchical, gradient-free selection layer that compresses and decompresses queries, keys, and values symmetrically while preserving left-to-right causality. The wrapper is removed near the end of training in a short recovery phase, so the deployed model runs vanilla attention with no architectural change at inference. Preliminary LLM experiments report faster total training time and lower final loss than full-attention baselines.
δ-mem
δ-mem augments a frozen full-attention model with a compact online associative-memory state. The state is a fixed-size matrix updated by delta-rule learning during generation, and its readout produces low-rank corrections to the backbone's attention output. There is no fine-tuning, no backbone swap, and no context extension.