AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
hLLM: Single Pass Decoding for Generative Reranking
Emil Laftchiev and colleagues at Meta Platforms observe that a generative ranker only ever needs to emit N ordinal values, and exploit that narrow output format to decode the whole ranking in a constant number of forward passes.
GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving
Qiankun Ma and colleagues point out that every KV compression method fixes the per-request budget in advance and only decides what to keep, then make capacity itself a runtime resource that grows on demand.
Hardware-Aware FP4 FlashAttention-4
Robert Hu shows that Blackwell FP4 tensor cores do not speed up attention on their own, because softmax conversion and on-chip dependencies dominate once the matrix products shrink, and gives separate forward and causal paths that recover the speedup.
SGD-KV: Summarization Guided KV Cache Compression
Zeyu Liu and colleagues present SGD-KV, which identifies attention heads specialized in hierarchical aggregation using a chunk-summarization diagnostic and allocates KV cache budget to them instead of applying a uniform heuristic.
Interface-Induced Trajectory Censoring
Wenbo Wang (City University of Hong Kong) shows that a reported tool-call rate can be exactly zero while the model emits well-formed calls the whole time, because the serving adapter censors the trajectory before the executor or scorer ever sees it.
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
Subham Sekhar Sahoo and colleagues introduce Uno, a class of diffusion-augmented LLMs that keeps an autoregressive model's exact distribution while drawing several tokens per step from it, giving lossless speedups without a draft model.
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Sergii Kozyrev and Davyd Maiboroda test the community intuition that Gated DeltaNet's recurrent state is too fragile for 4-bit, quantize all 496 linear layers of a hybrid 27B including the gates, and find it matches BF16.
Loom: Weaving Diagnostic Strands into Free-Text Consensus via Embedding-Space Reweighting
Ron Begleiter, Katya Egert Berg, Gilad Saban and Gil Shabat at NVIDIA present Loom, a deployed root cause analysis system that aggregates open-form hypotheses from modular heuristics in embedding space and spends exactly one LLM call per incident.
Runtime-Independent Persistent Agents: Preserving Identity, Memory, and Code Across Models, Harnesses, and Servers
We describe an agent by whatever model and harness it happens to run on, which works for one session and says very little about an agent running for months across a new model, a new harness, or a new machine. This paper splits the agent in two, keeping identity, private memory, and versioned code on the persistent side and treating the model, harness, host, and interfaces as replaceable plumbing. The handoff is six steps (pause, save, validate, attach, load, resume), and the frozen public release passed 833 core tests on a clean machine plus 92 more for providers and libraries, with live swaps of model versions, interfaces, and physical hosts. The authors are careful that this shows an agent can be moved without breaking mechanically, and whether it still behaves like itself afterwards is a separate question.
SkillZip Pro: Execution-Aware Dynamic Compression of Progressively Loaded Skills for Self-Evolving Agents
Xiaofan Bai and colleagues compress whole progressively loaded skill bundles rather than single prompts, removing content from a reference when the root or an environment contract already supplies it, while preserving every route.
Paritok-4B: Intent-Conditioned Context Compression for Coding Agents
Jiayu Shi and Luzhuo Chen release Paritok-4B, a 4B LoRA compressor that shrinks coding-agent context to a quarter of its size by extracting spans rather than paraphrasing them, with weights and data open under Apache 2.0.
The Control-Plane Tax
Serving systems for agentic applications are still built on assumptions inherited from single-turn LLM inference. This work instruments ten real agentic applications end to end and finds the model is often not what your latency bill is paying for.
Reason Wide, Not Deep
Reasoning modes beat non-reasoning modes on multi-step agentic tasks and charge a 3x to 6x output-token premium on every single episode. Much of that spend goes into re-deriving procedures the model already worked out on earlier episodes in the same domain, which means the cost is recurring by accident rather than by necessity.
Zero-Mem
Production memory stacks spend extra model calls on summarizing interactions, writing records, and reranking retrievals. Each of those calls costs tokens and latency, and the generated summaries quietly discard the evidence you later need. This work asks whether structured memory access requires generation at all.
Prompt-Induced Waste
Two prompts can request the same code change and produce the same correct patch while causing a coding agent to perform radically different kinds and amounts of work. This preregistered study measures that effect across 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real harnesses.
TokTier
Serving stacks cache prompt KV state while the front end still re-tokenizes the full request text on every call, and coding agents pay the most because each call resubmits a long transcript after a small append that can move token boundaries near the tail. Across 153,951 real agent calls at a 94.1% prompt-cache hit rate, tokenization grows from 10% to 64% of time to first token. TokTier is a stateful CPU and GPU tokenization service with one contract, namely that emitted token IDs always match full reference tokenization. For session continuations it re-tokenizes a small window around the append and splices only when a stable-boundary check passes, otherwise widening or falling back, and for calls without a reusable prefix it runs exact pre-tokenization and BPE on a GPU. Differential campaigns across 17 production tokenizer families covering 1.5e10 split checks show zero divergence, incremental repair takes 0.5 to 1.1 ms from 100K to 3M characters (up to 437x faster than HuggingFace), and median time to first token drops 16 to 34% under vLLM.
JAXBench
GPU kernel optimization has KernelBench to hillclimb on. TPUs had nothing, and the Pallas DSL is documented thinly enough that models mostly guess. Google, with Harvard and UC Berkeley, closes that gap and finds a clean lesson about context along the way.
Beyond AdamW
Higher-order optimizers have promised faster convergence than AdamW for a while, with computational cost and numerical stability as the standing objections. This NVIDIA work adapts them for large-scale pretraining, identifying instabilities in SOAP at large batch sizes and eliminating the loss spikes with per-step QR orthogonalization and improved preconditioning, then running a unified study of SOAP, Muon, and AdamW under update-RMS matching for fair learning rate transfer. On multi-billion-parameter models trained over trillions of tokens, SOAP and Muon consistently beat AdamW, and at batch sizes up to 100M tokens for next-token prediction they hold stability and quality while AdamW degrades. A layer-wise distributed optimizer compatible with Megatron-LM balances memory and hides communication without approximating the optimizer math.
The Self-Speculating Agent
Agents spend a large share of wall-clock time waiting on tool results. Speculation hides that latency by predicting and pre-executing the next call, but external draft models and cached traces model a different policy, so they miss too often to help. UC Santa Barbara and LinkedIn identify this speculator-agent gap and unify both roles in one model. It runs in agent mode to solve the task and in speculator mode to predict its next tool call from a partial trajectory, fully reusing the prefix KV cache. Joint agent-speculator reinforcement learning derives speculation targets from the agent's own rollouts and alternates updates between the two modes. Next tool-call Hit@1 rises from 44.1 to 61.2 for Qwen3-4B and from 48.9 to 66.3 for Qwen3.5-4B, with agent task success preserved.
When Is Routing Meaningful
LLM routers and mixture-of-agents systems get judged on accuracy and cost, both of which can look great while the router is doing nothing. This DeepMind-affiliated work argues that whether routing means anything depends on two properties that are orthogonal to accuracy.
LingBot-World 2.0
Most world models fall apart after a few seconds, smearing textures and warping geometry as errors compound frame to frame. LingBot-World 2.0 from Robbyant holds 720p at 60 fps for a full hour of interaction and ships fully open.
HOLA
Linear-attention and state-space models compress an entire prefix into a fixed-size state, buying constant memory but overwriting earlier facts when many key-value associations compete. HOLA gives linear attention a hippocampal complement, pairing a compressive recurrent state with a small exact memory to recover long-range recall.
Puzzle-75B
Bigger mixture-of-experts models keep winning on quality, but serving them at interactive latency is still hard. NVIDIA compresses the hybrid MoE Nemotron-3-Super into Puzzle-75B-A9B and roughly doubles interactive server throughput while holding quality.
The Harness Effect
As orchestration harnesses mediate every model call, this study asks how much the harness alone moves cost and performance. It ran 22 evaluation tasks across six foundation models, then changed only the orchestration layer while holding the models constant.