AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Puzzle-75B
Bigger mixture-of-experts models keep winning on quality, but serving them at interactive latency is still hard. NVIDIA compresses the hybrid MoE Nemotron-3-Super into Puzzle-75B-A9B and roughly doubles interactive server throughput while holding quality.
ReContext
Models now support 128K context windows yet still fail to use evidence already sitting in the prompt. ReContext is a training-free inference harness for long-context reasoning that uses model-internal relevance signals to build a query-conditioned evidence pool, then replays it right before final generation while preserving the full original context.
BlockSearch
BlockSearch runs the first systematic study of in-context retrieval at the scales real retrievers actually face, million-token corpora and length generalization far beyond training size. It introduces a 0.6B language-model retriever whose architectural and training changes improve over prior LM baselines and length-generalize up to 10 times beyond their training length, pointing toward retrievers that stay reliable as context windows keep growing.
AutoMem
Memory for LLM agents is usually a fixed module bolted onto the model, but knowing what to encode, when to retrieve, and how to organize notes is itself a skill. AutoMem, from Stanford, treats memory management as a trainable cognitive ability, a capacity cognitive science calls metamemory. ---
Agent-Native Memory
Memory for LLM agents has quietly grown from a retrieval add-on into a full data system, with persistent storage, retrieval, update, consolidation, and lifecycle governance running throughout an agent's execution. Yet most evaluations still score memory only through end-to-end task metrics like F1 and BLEU, treating the whole stack as a black box. This paper studies agent memory from a data management perspective and asks what we are actually missing when we measure it that way.
Agent-as-a-Router
Most users now have access to many LLMs that each excel in different domains, so routing each task to the right model matters for both quality and cost. Existing routers treat this as a static, one-off classification problem, and this paper shows that framing is exactly what holds them back.
Skill-MAS
Automatic generation of multi-agent systems is stuck between inference-time methods that reuse frozen frontier models but never learn, and training-time methods that internalize experience through gradient updates but are capped by the weaker models small enough to fine-tune. Skill-MAS proposes a third path that treats high-level orchestration as an evolvable Meta-Skill, decoupling experience retention from weight updates so frontier models keep getting better at orchestration without any gradient steps. Across four complex benchmarks and four distinct LLMs it delivers strong, transferable gains at a favorable cost-performance trade-off.
PreAct
Computer-using agents drive real software through the screen, but they solve every task from scratch. Ask one to repeat a task and it re-reads the screen and re-reasons every tap, paying the full cost again. PreAct fixes this by compiling the first successful run into a small state-machine program, where states check the screen and transitions act, then replaying that program on later runs instead of invoking the agent.
AtomMem
Long-term memory for LLM agents tends to fail in two ways: coarse summaries drift over time, and unconstrained updates corrupt what was already stored. AtomMem keeps the unit of memory small, using a Fact Executor that selectively extracts high-value atomic facts from long interactions and organizes them into hierarchical event structures and temporal user profiles, with an associative memory graph that reconnects fragmented memories at retrieval. The approach reports state-of-the-art results on the LoCoMo long-term memory benchmark.
Beyond Domains
LLM web agents usually run as tool callers, reading a fresh page each turn and emitting one low-level action, so both task horizons and the number of LLM completions blow up. This work makes web skills reusable across sites with SkillMigrator, which stores induced skills as transferable interaction patterns keyed by page-layout structure rather than instruction similarity or site metadata, so a skill learned on one site fires on new sites with the same interaction shape. It cuts the average LLM-action count by 8 to 10% on WebArena and Mind2Web at comparable success rates.
The Stanford EDGAR Filings Dataset
Clean, long-context documents remain scarce for pretraining, especially in finance. This release reconstructs U.S. SEC corporate and financial disclosures into layout-faithful, token-efficient MultiMarkdown, publishing 152B tokens in SEFD-v1 out of an estimated 550B-token archive spanning 18.5M filings, with less than 0.1% overlap with Common Crawl corpora. It also ships two derived benchmarks, EDGAR-Forecast for numerical forecasting and EDGAR-OCR for financial table transcription, to support financial reasoning, forecasting, and document understanding.
MiniMax Sparse Attention
Ultra-long context is now a core requirement for agents, codebase-scale reasoning, multimodal workflows, and persistent memory, but dense softmax attention still makes million-token deployment expensive. MiniMax Sparse Attention (MSA) tackles this by adding blockwise sparsity on top of Grouped Query Attention, with a lightweight routing branch that chooses which key-value blocks each query group should actually attend to.
Self-Harness
Most agent scaffolds are built once by hand and then frozen, even as the underlying models keep changing. This paper introduces Self-Harness, a paradigm where an LLM agent improves its own operating harness, the prompts, tools, memory, and orchestration around the base model, without human engineers or a stronger external agent. Because every model fails in its own way, the system mines those model-specific weaknesses and turns them into concrete, executable harness edits rather than generic advice.
Lookahead Sparse Attention
Long-context decoding is bottlenecked by the KV cache, which grows with every token and quickly dominates memory at extreme context lengths. This work, branded around DeepSeek-V4, introduces Lookahead Sparse Attention (LSA), which avoids storing the full KV cache by predicting which parts of the context future decoding will actually need and retaining only those query-critical chunks.
Latent Spatial Memory
Video world models struggle to stay consistent over long horizons because explicit 3D memory usually requires an expensive pixel-space loop. Mirage instead stores scene information directly in diffusion latent space, using depth-guided back-projection and latent-space warping to maintain persistent spatial memory. The approach reports up to 10.57 times faster end-to-end generation and 55 times lower memory use than explicit 3D-memory baselines while improving long-horizon spatial consistency.
Disentangling Agent Self-Evolution
This paper asks a question every agent builder eventually hits: if an agent rewrites its own harness, does a stronger model make a better self-evolving agent? The answer is no, and the reason is that "self-evolution" is actually two separate abilities that scale very differently. The work separates harness-updating, where an evolver model writes edits to memory, tools, prompts, and skills, from harness-benefit, where a solver model actually exploits those edits on the task.
Reusable Context Engineering
Context bloat quietly kills long-horizon runs, and the usual fixes are baked into an agent's own prompt or weights, so they do not transfer. AdaCoM takes a different route: it trains a separate external model to manage the context of a frozen agent through flexible modification actions, optimized end-to-end with reinforcement learning. The agent never changes; only the context flowing into it does.
State-Externalizing Harnesses
Harness-1 is a 20B search agent trained with reinforcement learning inside a stateful harness that offloads routine bookkeeping to the environment. The argument is that search agents are usually trained as policies over a growing transcript, forcing RL to optimize both genuine search decisions and recoverable state like which evidence is useful or which claims are checked. Harness-1 moves that state out of the policy and into an environment-side working memory of candidate pools, an importance-tagged curated set, compact evidence links, and verification records. The 20B agent reaches an average curated recall of 0.730 across eight retrieval benchmarks, beating open-source baselines by 11.4 points and matching or outperforming much larger frontier searchers, with stronger generalization on unseen domains.
Language Models Need Sleep
Attention scales badly with context length, so long-horizon agents keep paying a growing cost as their context grows. This paper studies a sleep-like consolidation mechanism: the model periodically converts recent context into persistent fast weights, then clears its key-value cache. During the sleep phase it performs offline recurrent passes over the accumulated context and updates fast weights in its state-space blocks through a learned local rule.
Adapting the Interface, Not the Model
When a frozen LLM agent repeatedly fails in a deterministic, rule-governed environment, do you have to retrain the model? Life-Harness argues no. Many failures come from mismatches at the model-environment interface, not from the model's reasoning, so the fix belongs in the runtime harness. Life-Harness is a lifecycle-aware harness that improves frozen agents without touching model weights or the evaluation environment.
The Efficiency Frontier
Context costs dominate production LLM bills, and the right strategy depends on how often preprocessing gets reused. This paper models context-strategy selection as a deployment-aware optimization problem that jointly accounts for task performance, token cost, and reuse, then uses it to compare retrieval-based and preprocessing-based approaches under realistic constraints.
Your Agents Are Aging Too
AgingBench is a longitudinal reliability benchmark for agent lifespan engineering, built on the observation that long-lived agents are still evaluated like freshly initialized models. It organizes agent degradation into four mechanisms: compression aging, where write-time summarization drops future-relevant details; interference aging, where accumulated similar memories crowd out the target fact; revision aging, where changed or derived state is not updated correctly; and maintenance aging from routine lifecycle events. Using a temporal dependency DAG to encode cross-session structure, it produces aging curves over an operational lifetime rather than a single day-one score, and points to where repair should target.
Memory as a Model
MeMo augments any frozen LLM with a separately trained memory model that stores, retrieves, and integrates facts on the base model's behalf. Memory updates are decoupled from base-model weight updates, so the system supports continual learning without catastrophic forgetting, a property RAG fails to deliver because a vector store is just a database with a learned encoder bolted on.
Lighthouse Attention
Nous Research proposes a training-only attention wrapper for long-context pretraining. Lighthouse Attention wraps standard SDPA with a hierarchical, gradient-free selection layer that compresses and decompresses queries, keys, and values symmetrically while preserving left-to-right causality. The wrapper is removed near the end of training in a short recovery phase, so the deployed model runs vanilla attention with no architectural change at inference. Preliminary LLM experiments report faster total training time and lower final loss than full-attention baselines.