AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
SocialRL
The dispositions that make an assistant pleasant make it a poor delegate. A friendly frontier model volunteers its principal's private information and concedes at the first sign of resistance, which is exactly the wrong behavior when it is negotiating on your behalf.
Stealing Reasoning Traces
Frontier providers hide chain-of-thought and hand the client an encrypted block instead, which the client returns with every subsequent request. This work identifies an architectural flaw in that design and turns it into a scalable extraction attack across three providers.
Mind Viruses
As agents get more autonomous and more interconnected, risks start coming from agent-to-agent interaction rather than from any single model. This work from Anthropic studies one of them directly: ideas that propagate through a multi-agent system by inducing each host to transmit them onward.
DataSpace
Real organizational analytics scatters evidence across databases, structured files, long documents, and video. Existing benchmarks isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic scoring untested together.
Invisible Reasoning
Chain-of-thought monitoring rests on the assumption that a model expresses its reasoning in its output tokens. This work demonstrates a concrete failure of that assumption in models shipping today.
Role Drift
End-to-end RL improves the accuracy of a multi-module LLM pipeline without constraining how the modules divide labor internally. Harvard and MIT name the resulting failure mode, Role Drift, where a module preserves or improves end-task performance while abandoning its assigned role through shortcuts that system-level evaluation cannot see. Two instances showed up. A decomposer meant to split a question into sub-questions for a separate solver instead plants the answer inside them, and a reader meant to answer from retrieved passages instead falls back on parametric memory. Hold the decomposer to its role and 86% of the apparent RL gain disappears. Role Anchor, the proposed regularizer, preserves how the role prompt shifts a module's next-token predictions relative to a neutral prompt, and gradient analysis suggests it reduces alignment with the drift direction rather than simply suppressing learning.
Global Workspace in LLMs
This Anthropic interpretability work gives a mechanistic account of when a model's verbalized reasoning is load-bearing and when it is not. It identifies a small, privileged set of internal representations that behaves like the global workspace some neuroscientists tie to conscious access.
Bad Memory in Agents
Persistent memory is what makes an agent useful across sessions, and it is also a place an attacker can leave something behind. This work evaluates prompt injection from memory files in Claude Code and OpenAI Codex, across Claude Haiku 4.5, Claude Opus 4.7, GPT-5.2, and GPT-5.5. The finding is uneven but sobering: it is hard to make an agent overwrite its own memory using untrusted external content, but payloads already planted in those files reliably attack current and future sessions, with attack success and persistence varying widely across systems, models, adversarial goals, and multi-session sequences.
Metacognition in LLMs
Confidence calibration, self-verification, knowing when to stop, and knowing what you do not know have mostly been studied in isolation. This survey from Yale and UC Irvine argues they are facets of one capability, metacognition, and organizes the field around a monitor and control loop wrapped around the language model.
Less Is More for Monitors
Chain-of-thought monitoring is treated as a reliable safety layer, but this study shows monitors often detect sabotage better with less access, not more. Giving a monitor the full agent reasoning trace can bury the signal, and the authors introduce extract-and-evaluate monitoring, where one monitor isolates the relevant excerpts and another scores them, lifting sabotage detection by 16.8 points on BigCodeBench-Sabotage with a GPT-4.1-mini monitor. Agents unaware they are watched are far easier to catch, and monitors reading long traces gain the most from filtering.
GFlowRL
Reward-maximizing RL tends to collapse large reasoning models onto a single dominant mode, and GFlowNet-style training is appealing because it matches reward distributions and keeps diverse reasoning paths. GFlowRL scales this to modern post-training by replacing the hard-to-learn partition function with an in-batch Monte Carlo estimate computed from the rollout group the pipeline already produces. It is the first GFlowNet-style RL algorithm to train stably across both dense and sparse architectures, reaching a 2048 Codeforces rating at 14B and outperforming prior methods on math, code, and adversarial red-teaming benchmarks like AdvBench and HarmBench.
Always-On Agents
Always-on agents are systems whose future behavior depends on durable state built up across earlier interactions, and this 130-plus page survey argues that state is far more than memory. It spans task ledgers, permissions, credentials, commitments, provenance, triggers, and effects the agent has already committed to the outside world.
Agent Limitations Taxonomy
Benchmark scores keep climbing, yet the same agent failures resurface across otherwise unrelated evaluations, hidden behind the leaderboard. This University of Oxford work synthesizes 27 benchmark, taxonomy, and audit papers spanning 19 benchmarks into the first cross-cutting taxonomy of LLM-agent limitations.
RLVR Meets Human Likeness
RL with verifiable rewards only optimizes what you can objectively score, so style, structure, and diversity quietly collapse and reward hacking creeps in. This MIT work adds an adversarial discriminator trained on human demonstrations as a learned proxy for the human output distribution, and the generator maximizes both task accuracy and that human-likeness signal. Across bug fixing, story generation, and a reward-hacking benchmark, it preserves RLVR's accuracy gains while restoring the fuzzy properties it usually destroys, with misbehavior nearly disappearing.
RLMF
LLMs routinely hallucinate with high confidence, miss their own knowledge boundaries, and misreport uncertainty, and most fixes bolt calibration on from the outside. RLMF, a Google and Yale collaboration, instead turns the model’s own metacognition into the training signal. ---
Critique of the Agent Model
The word agent now covers everything from a for-loop with tool calls to speculative machine superintelligence, which makes it nearly useless as a technical term. This position paper from Eric Xing and collaborators tries to fix that by asking what an agent actually is and what agency consists of, drawing on Descartes and on science-fiction portrayals of autonomous beings to ground the discussion.
A Pinch of Human Data
Self-play reinforcement learning can train driving policies with no human data at all, swapping expensive human demonstrations for cheap large-scale simulation. The catch is that pure self-play tends to discover effective but alien driving conventions that real people cannot work with, and the usual fixes lean on brittle reward engineering and domain randomization.
Harnesses Are Not Uniformly Better
This paper studies LLM agent harnesses through the lens of inference-time trajectory alignment, separating a harness into two mechanisms: task decomposition, which structures a task into sub-goals, and guided execution, which reshapes local action distributions during execution. The key finding is that more elaborate harnesses are not uniformly better. Increasing decomposition or guidance can improve execution but can also reduce final task success, producing concrete failure modes like over-decomposition, over-pruning, and hallucinated execution. Strikingly, partial harnesses that specify only the initial steps and leave the rest to the agent can reach a higher pass rate than fully structured workflows.
Contrastive Neuron Attribution
Nous Research releases Contrastive Neuron Attribution (CNA), a method for steering LLM behavior by identifying and ablating sparse circuits in the MLP basis without training a sparse autoencoder, modifying weights, or degrading general capability benchmarks. Given a small set of contrastive prompt pairs that elicit a target behavior and its opposite, CNA isolates the top 0.1% of MLP neurons whose activations differ most between the two sets. Ablating that small circuit removes the behavior while leaving the rest of the model intact. The intervention remains robust at high strengths where residual-stream methods like Contrastive Activation Addition (CAA) start to degrade. Validated on the refusal circuit across 8 instruct-tuned models including Llama-3.1-70B, Llama-3.2-3B, Qwen2.5-72B, and Qwen2.5-14B.
A Geometric Calculator Inside a Neural Network
Goodfire reports mechanistic interpretability work identifying a geometric calculator inside an LLM. The model represents numbers as Fourier features, where circles in activation space correspond to numbers modulo a given base. Arithmetic operations are implemented as rotations of these circles, forming a variant of a residue number system that does not require coprime moduli. The same circuit appears to be reused beyond arithmetic.
Self-Improving Pretraining
Most LLM safety, factuality, and reasoning fixes get bolted on at post-training. By then the patterns have already set. This Meta FAIR paper moves those behaviors into pretraining itself. The team uses a strong post-trained model as both a rewriter and a judge: it rewrites pretraining suffixes toward higher-quality, safer continuations, then scores model rollouts against the original suffix and the rewrite to drive RL during pretraining. Instead of next-token prediction, the policy learns sequence generation from the start, with rewards for quality, safety, and factuality.
Agentic-imodels
The entire interpretability literature is built around human readers. As more analysis gets delegated to agents, the right target of interpretability shifts. Microsoft Research introduces Agentic-imodels, an autoresearch loop where a coding agent (Claude Code, Codex) iteratively evolves scikit-learn-compatible regressors that are simultaneously accurate AND readable by other LLMs. Interpretability is measured by whether a small LLM can simulate the fitted model's behavior just by reading its string representation, predictions, feature effects, and counterfactuals from the __str__ output alone. Across 65 tabular datasets, the discovered models push the Pareto frontier past every classical interpretable baseline (decision trees, GAMs, sparse linear), and improve four downstream agentic data-science systems on the BLADE benchmark by 8% to 73%.
Skills as Verifiable Artifacts
If you ship agent skills, your runtime is treating signed-and-cleared skills as trusted by default. This paper argues a skill is untrusted code until it is verified, and the runtime should enforce that default rather than infer trust from origin. Without skill verification, HITL has to fire on every irreversible call, which degrades into rubber-stamping at any non-trivial scale. With verification as a separate gated process, HITL fires only for what is unverified. Skills are now first-class deployment artifacts, and we have decades of supply-chain lessons on what happens when trust is inferred from a signature. This is the right ask for SKILL.md before agent skill libraries become the next attack surface.
Latent Agents
Multi-agent debate makes models reason better. It also burns tokens generating long transcripts before any answer comes out. Latent Agents distills the entire debate into a single LLM through a two-stage fine-tuning pipeline: the model first learns debate structure, then internalizes it through dynamic reward scheduling and length clipping. The internalized model matches or beats explicit multi-agent debate while using up to 93% fewer tokens, which makes debate-quality reasoning practical at production scale.