AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Discover and explore top AI papers with Claude Code or Codex
npx @dair-ai/mcp setup
Confucius Code Agent
Confucius Code Agent (CCA) is a software engineering agent designed to operate on large-scale codebases. Built on the Confucius SDK, it introduces a three-axis design philosophy separating Agent Experience (AX), User Experience (UX), and Developer Experience (DX) to enable robust multi-step reasoning and modular tool use.
End-to-End Test-Time Training for Long Context
This paper reframes long-context language modeling as a continual learning problem rather than architecture design. TTT-E2E uses a standard Transformer with sliding-window attention that continues learning at test time via next-token prediction, compressing context into its weights rather than storing all key-value pairs.
Geometric Memory in Sequence Models
This paper identifies a dramatically different form of how deep sequence models store factual information called geometric memory, contrasting with the traditional associative memory view. Models synthesize embeddings encoding global relationships between all entities, even ones that never co-occur in training.
Test-Time Training for Long-Context LLMs
This paper shows that long-context LLMs can access millions of tokens but often fail to meaningfully use that information. The authors propose query-only test-time training (qTTT), which adapts models during inference through targeted gradient updates rather than generating more thinking tokens.
Sophia
Sophia introduces System 3, a meta-layer beyond traditional dual-process theory that enables LLM agents to maintain persistent identity and align short-term actions with long-term goals. The framework achieves 80% reduction in reasoning steps for recurring operations and 40% performance improvement on high-complexity tasks.
DeepCode
DeepCode is a fully autonomous framework for synthesizing complete codebases from scientific papers despite LLM context limitations. It treats repository synthesis as a channel optimization problem, achieving state-of-the-art on PaperBench and outperforming commercial tools like Cursor and Claude Code.
LatentMAS
LatentMAS introduces a framework enabling language model agents to collaborate directly within a continuous latent space rather than relying on text-based communication. By using last-layer hidden embeddings and a shared latent working memory, agents preserve and transfer internal representations without information loss from text serialization.
Nested Learning Framework
Google Research introduces Nested Learning (NL), a paradigm representing models as nested optimization problems where each component has its own context flow, revealing that deep learning methods compress context and explaining how in-context learning emerges. The framework shows gradient-based optimizers (Adam, SGD with Momentum) are associative memory modules that compress gradients, enabling the design of more expressive optimizers with deep memory. The HOPE architecture, combining self-modifying sequence models with continuum memory systems, achieves strong results on language modeling (15.11 WikiText perplexity at 1.3B parameters), outperforming Transformers and modern recurrent models.
Context Engineering 2.0
Researchers from SJTU, SII, and GAIR trace the 20+ year evolution of context engineering, reframing it as a fundamental challenge in human-machine communication spanning from primitive computing (Era 1.0) to today’s intelligent agents (Era 2.0) and beyond. It defines context engineering as systematic entropy reduction where humans preprocess high-entropy contexts into low-entropy machine-understandable representations. This gap narrows as machine intelligence increases.
Enhancing Long-Term Memory in LLMs
Researchers from the University of Alberta and UMass Amherst introduce BEAM, a new benchmark for evaluating long-term memory in LLMs with conversations up to 10M tokens, and LIGHT, a framework that enhances memory performance through three complementary systems.
AgentFold
AgentFold introduces proactive context management for long-horizon web agents, addressing context saturation through dynamic “folding” operations that balance detail preservation with efficient compression. The 30B parameter model outperforms dramatically larger competitors while achieving state-of-the-art results on web browsing benchmarks.
Kimi Linear
Kimi Linear introduces a hybrid linear attention architecture combining Kimi Delta Attention (KDA) with periodic full attention layers at a 3:1 ratio, achieving superior performance over full attention while reducing KV cache by 75% and delivering 6× faster decoding at 1M context. KDA extends Gated DeltaNet with fine-grained channel-wise gating and specialized Diagonal-Plus-Low-Rank matrices, enabling more effective RNN memory management while maintaining hardware efficiency through optimized chunkwise algorithms that substantially reduce computation versus general DPLR formulations.
Continual Learning via Sparse Memory Finetuning
Meta AI researchers address catastrophic forgetting in language models through sparse memory finetuning, updating only memory slots most activated by new knowledge while achieving 89% less performance degradation than standard finetuning.
Bayesian Influence Functions for Hessian-Free Data Attribution
Classical influence functions struggle with deep neural networks due to non-invertible Hessians and high-dimensional parameter spaces. This work introduces the local Bayesian influence function (BIF), which replaces Hessian inversion with loss landscape statistics estimated via stochastic-gradient MCMC sampling.
Reasoning over Longer Horizons via RL
The authors show that you can scale long-horizon reasoning without step labels or heavy scaffolding. They synthesize long problems by chaining easy ones, then train with outcome-only rewards under a length curriculum. The result: large gains on both in-domain chains and harder out-of-domain math and long-context tasks.
The Markovian Thinker
A new RL thinking environment that keeps an LLM’s effective state constant by chunking long chains of thought and carrying over only a short textual state between chunks. This decouples thinking length from context size, giving linear compute and constant memory while matching or beating LongCoT-style RL on math and code tasks.
Abstract Reasoning Composition
UC San Diego and UMD propose ArcMemo, a test-time memory framework that distills reusable concepts from solution traces, stores them in natural language, and retrieves a relevant subset on future queries. Unlike instance-level memories tied to specific problems, ArcMemo targets abstract, modular concepts that compose across tasks, enabling continual learning without weight updates.
mem-agent
mem-agent is a 4B-parameter LLM trained with GSPO reinforcement learning to develop persistent memory using a scaffold of Python tools and markdown files. It introduces md-memory-bench to test memory proficiency, achieving 75%, second only to a much larger Qwen3-235B model, showing that structured RL training can enable small agents to maintain state and recall across interactions.
Artificial Hippocampus Networks
Artificial Hippocampus Networks add a fixed-size recurrent memory to sliding-window Transformers, compressing evicted KV into RNN-like states (Mamba2/DN/GDN) trained via self-distillation for long-context efficiency with constant cache and near-linear compute. On LV-Eval 128k, Qwen2.5-3B + AHN (+0.4% params) cuts FLOPs 40.5% and cache 74% while raising average from 4.41 to 5.88, though exact-recall NIAH tasks still favor full attention.
Fine-tuning LLM Agents without Fine-tuning LLMs
A memory‑based learning framework that lets deep‑research agents adapt online without updating model weights. The agent is cast as a memory‑augmented MDP with case‑based reasoning, implemented in a planner–executor loop over MCP tools. It sets top validation results on GAIA and delivers strong scores on DeepResearcher, SimpleQA, and HLE.
Memory-R1
A framework that teaches LLM agents to decide what to remember and how to use it. Two RL-fine-tuned components work together: a Memory Manager that learns CRUD-style operations on an external store and an Answer Agent that filters retrieved memories via “memory distillation” before answering. Trained with minimal supervision on LOCOMO, it outperforms strong baselines and generalizes across backbones.
M3-Agent
A framework for agents that watch and listen to long videos, build entity-centric memories, and use multi-turn reasoning to answer questions. M3-Agent stores both episodic details and distilled semantic knowledge in a multimodal memory graph, then learns a retrieval-reasoning policy with RL. The authors also introduce M3-Bench, a long-video QA benchmark with robot-view and web videos. Results show consistent gains over strong prompting baselines.
OdysseyBench
A new benchmark and data‑generation pipeline to test agents on realistic, multi‑day office tasks across Word, Excel, PDF, Email, and Calendar. It introduces OdysseyBench (two splits) and HOMERAGENTS (an automated multi‑agent generator), with evaluations showing large gaps between human and model performance and clear benefits from semantically compressed memory.
Efficient Attention Mechanisms
This survey reviews linear and sparse attention techniques that reduce the quadratic cost of Transformer self-attention, enabling more efficient long-context modeling. It also examines their integration into large-scale LLMs and discusses practical deployment and hardware considerations.