Model or Harness
Agent evaluations mostly report system-level outcomes, so a failed run leaves the repair unassigned. The same visible failure might call for model post-training, harness engineering, environment redesign, or benchmark repair, and outcome labels cannot separate those cases.
Zero-Mem
Production memory stacks spend extra model calls on summarizing interactions, writing records, and reranking retrievals. Each of those calls costs tokens and latency, and the generated summaries quietly discard the evidence you later need. This work asks whether structured memory access requires generation at all.
Sample More Reflect Less
Methods that make a model criticize and rewrite its own answer nearly all generate far more text than a single chain of thought. Since generating more text raises accuracy on its own, a reported gain leaves open whether the method's idea is what helped. This paper reruns the comparison as a designed experiment.
Harness-R1
Agents accumulate interaction trajectories during deployment and then leave them unused, because their behavior stays fixed. Those trajectories can improve the harness that constructs context, mediates tools, validates actions, and recovers execution, and this work makes that editing a learned capability.
DataSpace
Real organizational analytics scatters evidence across databases, structured files, long documents, and video. Existing benchmarks isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic scoring untested together.
Prompt-Induced Waste
Two prompts can request the same code change and produce the same correct patch while causing a coding agent to perform radically different kinds and amounts of work. This preregistered study measures that effect across 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real harnesses.
Rehearse
Autoresearch loops propose changes, run full training jobs, and keep whatever improves the metric. Their efficiency depends on judging, before spending a run, whether a proposed modification is likely to work, and this paper studies how that judgment holds up over a trajectory.
ContinualSkillBench
Skill libraries are shipping in agent harnesses on the assumption that writing skills down compounds, and this benchmark tests that assumption directly. ContinualSkillBench covers five domains, each with 100 interconnected subtasks ordered by increasing difficulty and built with deliberate opportunities for cross-task skill reuse. Sequential execution generally improves performance, though the gains vary substantially across models and domains, and maintaining an explicit skill library performs comparably to plain in-context learning on average. Much of the improvement comes from adapting to prior context and feedback rather than from reusable skill abstraction, though explicit skills still pay off selectively on tasks needing reusable procedures or precise outputs. There is a useful diagnostic buried in the results. Less capable models accumulate larger, more fragmented collections of task-specific skills, which is what failed abstraction looks like from the outside.
MerchantBench
Agent benchmarks tend to focus on bounded tasks with immediate success criteria, which flatters systems that cannot hold a plan for a month. MerchantBench targets long-term coherence instead, running a 365-day order-level e-commerce simulation grounded in 98,843 real product records with 26 tools for agent interaction. Agents handle product sourcing, listing and pricing control, cash-flow management, and feedback arriving at wildly different delays, with promptly observable supplier events coupled to delayed downstream order outcomes so earlier decisions must be revisited. Scoring runs on cumulative net assets, so incoherence compounds rather than averaging out. Across eight LLMs under two agent frameworks and 48 runs of 365 simulated days each, the best configuration reaches only 27.3% of the mean final net assets achieved by human participants.
TokTier
Serving stacks cache prompt KV state while the front end still re-tokenizes the full request text on every call, and coding agents pay the most because each call resubmits a long transcript after a small append that can move token boundaries near the tail. Across 153,951 real agent calls at a 94.1% prompt-cache hit rate, tokenization grows from 10% to 64% of time to first token. TokTier is a stateful CPU and GPU tokenization service with one contract, namely that emitted token IDs always match full reference tokenization. For session continuations it re-tokenizes a small window around the append and splices only when a stable-boundary check passes, otherwise widening or falling back, and for calls without a reusable prefix it runs exact pre-tokenization and BPE on a GPU. Differential campaigns across 17 production tokenizer families covering 1.5e10 split checks show zero divergence, incremental repair takes 0.5 to 1.1 ms from 100K to 3M characters (up to 437x faster than HuggingFace), and median time to first token drops 16 to 34% under vLLM.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack