AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
SemVerBench: Benchmarking LLM Comprehension of Version-Constraint Resolution Semantics
Qibai Chen (independent researcher) and Zeming Liu (Brown University) measure how well frontier LLMs resolve package version constraints across npm, PEP 440 and Cargo, and find predictable rule-specific blind spots that tool delegation removes.
BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
Shenghan Zheng and Christophe Hauser (Dartmouth College), with Dawn Song (UC Berkeley) and collaborators at Amazon, BenchFlow and several universities, build BenchShield, an instrumentation layer that detects reward hacking in agent benchmarks from a formal model of each evaluation's reward-relevant events.
What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead
Haseeb Mohammed Afsar (independent researcher) probes a seeded random sample of 400 servers from the 24,135-server MCP registry and finds that fewer than half start, and that popular tool-use benchmarks contain far more duplicated tools than real MCP servers do.
Fork Where the Model Changes Its Mind: Belief-Shift Branching for Tree-Structured Reinforcement Learning
Bin Lei and colleagues at Salesforce AI Research and the University of Minnesota place the forks of tree-structured RLVR rollouts at the step where the model's answer belief shifts most, which gives step-level credit where the outcome is still undecided.
Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents
Minghao Guo, Meng Cao and colleagues (MBZUAI and USTC) build Mr.LHDR, a deep research benchmark whose questions require long chains of dependent, multimodal evidence, and find that the best system fully completes only about a third of them.
Negative Self-Distillation: Learning to Reason by Avoiding Flaws
Rongcan Pei, Yu Meng and colleagues (University of Virginia) replace on-policy self-distillation's imitation of privileged solutions with Negative Self-Distillation, which pushes the model away from a self-generated flawed reasoner and needs no ground-truth answers.
Decoupling Readiness from Release for Tail-Aware Scheduling of Agentic LLM Workflows
Bochao Feng, Jidong Zhai and colleagues (USTB, Qiyuan Laboratory and Tsinghua) schedule agentic LLM workflows by deciding when ready turns are released to the inference engine, instead of releasing them as soon as they are ready, and cut P95 workflow time by up to 3.50x.
Rethinking Verbalized Confidence for LLM-as-a-Judge: A Compatibility Shift on Post-2025 Proprietary Models
Yu-Chung Hsiao (Cisco Systems) shows that on post-2025 proprietary models, verbalized confidence is a more robust soft score for LLM-as-a-Judge than log-probabilities, which reverses the standard advice.
When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination
Karan Parekh, Sanjana Pendyala Ravinder, Sana Mhapsekar and Medina Maloku (University of North Texas) plant 450 known contaminants across 150 academic papers and show that an LLM auditor's detection collapses as batch size grows, and that the failure mode at scale is fabrication rather than abstention.
Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability
Arnab Chattopadhayay and Debdipta Halder (independent researchers) place a Bayesian belief tracker outside the LLM and show it only the posterior over latent states, never the raw action-observation log, which makes the pair a sound Markov policy on the belief MDP.
Fortunate Recall: Ontology-Driven Memory Lifecycle Management for Persistent Coherence in LLMs
Ansuman Mullick and Eray Tuzun (Bilkent University) classify personal facts into a behavioral ontology and apply category-specific retention policies as deterministic functions over LLM-extracted metadata, then locate through ablation which half of the design produces which gain.
Strangers to Themselves: What Language Models Say About Themselves Is Generic
Phil Blandfort (Predictably Weird) and Urja Pawar (independent) turn model self-knowledge into a prediction test across nine behavioral evaluations, and find a model's self-report predicts its own behavior no better than a question about AI agents in general.
Can AI Agents Detect and Repair Artifact Drift in Network Experiments?
Tianzhu Zhang (Nokia Bell Labs), Weichen Tao (Telecom Paris), Changgang Zheng (Nanjing University) and colleagues define artifact integrity as a property an agent must preserve, and build NetArtifactBench to measure whether agents can repair inconsistent experiment records without breaking supported claims.
JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
Zixiang Chen, Yuheng Lu and colleagues at Beihang University introduce JarvisGUI, a benchmark that evaluates GUI agents on workflows spanning Android, Windows and Ubuntu, where intermediate results must move between devices.
UnitBoost: Managing Compound LLM Systems with a Merge Operator, Not a Model
Xing Zhang, Guanghui Wang, Yanwei Cui, Mengdie Flora Wang and Peiyang He (AWS Generative AI Innovation Center) replace the generative manager in a compound LLM system with a defined operator, and show it beats generative managers on three held-out benchmarks.
KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
Xi Shi and Qian Lou (University of Central Florida) build KVShareArena, a benchmark for reusing KV caches when the reused text is not a prompt prefix, which is the case for retrieval-augmented servers and for multi-agent coordinators reading reports written by other agents.
Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward
Eshwar Reddy M (Testsigma) and Sourav Karmakar (Intuit India) argue that the constraint on reasoning RL outside formal domains is the absence of a scalable sound reward, derive the exchange rate between verifier quality and test-time compute, and measure it against executable ground truth.
ROAM: Robust Organization of Atomic Memories for Agents through Semantic Relations
Jianjie Zheng, Peng Lai, Sijie Cheng and Guanhua Chen (SUSTech, Tsinghua, RayNeo, Deepexi) propose ROAM, which classifies each incoming-versus-stored memory pair by semantic relation before deciding what to store, instead of asking an LLM to add, update, delete or rewrite in one step.
Show-Harness: Just a VLM Agent Can Play Robots
Yanzhe Chen, Zechen Bai, Kevin Qinghong Lin and Mike Zheng Shou (Show Lab, NUS) present Show-Harness, a semantic action interface that lets a VLM control a robot directly, with embodiment-specific interpreters grounding each discrete action unit deterministically.
TRACE: Training Reasoning Agents for Causal Exploration with Synthesized Rewards
Rui Sun, Zhan Shi and Bing He (independent researchers) train diagnostic reasoning agents by sampling an intervention, injecting it into a simulator, and generating the observations it would produce, so the hidden intervention supplies an oracle label for a task where real ground truth would require expert investigation.
Stable Answers, Unfinished Reasoning: Why Self-Consensus Is Not a Safe Early-Exit Signal
Yunxiang Mo, Donghao Zhao (HKUST) and Hejia Geng (University of Oxford) preregister a sweep of 3,520 self-consensus early-exit rules and find that none clears three acceptance gates, because agreement measures answer persistence rather than reasoning termination.
The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents
Benjamin Gruenbaum, Doron Porat, Assaf Natanzon and colleagues at Eon generate a complete fictional company, including simulators of Salesforce, Zendesk, Slack and Gong, so enterprise agent answers can be graded exactly against computed answer keys.
Introducing Consort: A Spec-First Agent Framework for Enforced, Test-Driven Development on Live Database Branches
Kevin Hartman (Databricks) introduces Consort, a spec-first agent framework whose engineering discipline is enforced by a deterministic orchestrator and immutable tests the agent runs inside but cannot edit.
RobustSGPO: Search-Space Control for Agent Harness Evolution
Zibo Zhao, Jijun Shi, Ruiming Tang, Wenwu Ou and Kun Gai (Wuhan University and Kuaishou Technology) add explicit control over edit scope and restart point to semantic-gradient prompt optimization for agent harnesses, and measure it over 7,350 candidate attempts.