Skaling
Standard neural scaling laws assume model size and training data act on loss independently. That assumption bakes in a cross-derivative of exactly zero, and it is why the Chinchilla form drifts at the data-scarce and heavy-overtraining edges of the grid, which is exactly where deployment now happens.
Stealing Reasoning Traces
Frontier providers hide chain-of-thought and hand the client an encrypted block instead, which the client returns with every subsequent request. This work identifies an architectural flaw in that design and turns it into a scalable extraction attack across three providers.
Mind Viruses
As agents get more autonomous and more interconnected, risks start coming from agent-to-agent interaction rather than from any single model. This work from Anthropic studies one of them directly: ideas that propagate through a multi-agent system by inducing each host to transmit them onward.
Catastrophic Remembering
Agentic coding READMEs grow without bound in real repositories, stopping only when the repo retires or someone rewrites the file wholesale. This paper traces the cause to imperfect recall and gives the phenomenon a name that inverts the one continual learning is organized around.
The Bitter Lesson of Tool Calling
Tool calling is a design choice and the default choice is JSON. For code-capable models, exposing tools as code instead lets calls chain and parallelize naturally, but nobody had run the comparison on an established benchmark across model generations under realistic conditions.
Reason Wide, Not Deep
Reasoning modes beat non-reasoning modes on multi-step agentic tasks and charge a 3x to 6x output-token premium on every single episode. Much of that spend goes into re-deriving procedures the model already worked out on earlier episodes in the same domain, which means the cost is recurring by accident rather than by necessity.
Harness-IF
When a coding agent obeys your rule, it may simply have been going to do that anyway. Existing instruction-following benchmarks cannot tell compliance from coincidence because they concentrate rules in the user turn, while coding-agent benchmarks only score final task success.
Lost in Compaction
Context compaction is now standard in long-running agent systems, and it silently drops the instructions users most expect to persist. This work names that class, Session Constraints, instructions like "do not delete any emails until I confirm" meant to bind behavior for the rest of a session, and introduces COMPINT to evaluate compactors across multi-turn chat, agentic trajectory, and long-horizon research. Current compactors retain only 17% of injected constraints on average, and most leave the task worse off than running it without compaction at all. Retention swings with the compactor, the prompt, the context length, the phrasing, and where the constraint was injected, which is what makes the loss structural rather than a quirk of one setup. The fix is small and does not touch the compactor or the model: an SC-aware extractor running alongside as a plug-and-play module recovers over 90% retention in all three scenarios.
Cracks in the Foundation
You might assume architectural variations within the dense transformer paradigm barely move accuracy, and in the short-context setting you would be right. This work shows four minor decisions, normalization, GQA, pretraining context length, and sliding window attention, each made by at least one of the Olmo, Llama, and Qwen dense families, have a compoundingly negative effect on long-context extensibility. Any one alone is minor, but combining three or more drops downstream long-context performance by up to 47%, and none of it is detectable from short-context loss or validation sets, which is precisely how these choices survive into shipped models. Applying context extension early in pretraining exposes the problem cheaply. After over 170,000 GPU hours the authors release OlmPool, 26 comparable 7B models with checkpoints before and after extension, including several architectures that beat the Llama 3 architecture on long-context extensibility.
CEDAR
Complex systems research models feedback-driven phenomena from population dynamics to economic policy, and its central open problem is that nobody can predict how feedback structure gives rise to emergent behavior, which makes goal-directed design very hard. CEDAR, from Sakana AI, attacks that with LLM agents running Monte Carlo Tree Search over the space of feedback structures rather than tuning parameters on a fixed one. Systems are represented as a restricted runnable subset of Python with domain-specific primitives so the models can edit dynamics directly, an LLM Judge scores emergent behavior against the stated goal as a fitness function, and an LLM Editor proposes variants as a variation operator. The formalization is an MCTS variant with an LLM-parameterized transition kernel and value function, which preserves solution diversity while searching, and the LLM-based interpretability makes it possible to read back how a structural change produced the behavior.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack