BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
Shenghan Zheng and Christophe Hauser (Dartmouth College), with Dawn Song (UC Berkeley) and collaborators at Amazon, BenchFlow and several universities, build BenchShield, an instrumentation layer that detects reward hacking in agent benchmarks from a formal model of each evaluation's reward-relevant events.
Ask this paper
Problem: Agents in interactive benchmarks can raise their measured score by exploiting the reward path instead of solving the task. Current defenses are task-specific patches, prompt instructions or post-hoc detectors, and none gives reusable evidence that a run stayed within bounds.
Lifecycle model: Detection is grounded in a finite model of the evaluation's reward-relevant events inside the benchmark infrastructure.
Static and runtime analysis: A phase-aware taint analysis finds reward-hacking paths before a run, and a runtime counterpart attributes actual agent behavior from infrastructure-side evidence.
Dataset: BenchShield Trajectories holds 456 human-adjudicated trajectories drawn from more than 31,000 public agent runs across three benchmarks.
Results: Against an agentic hackability scanner, full-chain recall rises from 23-94% to 77-100% and same-vector coverage from 16-56% to 43-78%, per-task cost falls up to 65%, and runtime detection reaches 96% accuracy.
Abstract
LM-agent benchmarks increasingly function as interactive evaluation infrastructure. Agents observe state, call tools, modify workspaces, submit artifacts, and receive rewards from outcome procedures. This interactivity makes evaluations vulnerable to reward hacking: an agent improves its measured score by exploiting the reward-relevant trajectory instead of solving the intended task. Existing defenses rely largely on task-specific patches, prompt instructions, or post-hoc detectors. They do not provide reusable evidence that a concrete run remained within its intended evaluation boundary. This paper presents BenchShield, a model-backed instrumentation layer for reward integrity in LLM-agent evaluation. BenchShield grounds detection in a finite lifecycle model of an evaluation's reward-relevant events. Within the benchmark infrastructure, two complementary analyses operate over this model. A static, phase-aware taint analysis exposes reward-hacking paths before a run. Its runtime counterpart uses infrastructure-side evidence to attribute concrete agent use and emit evidence-backed claims. We construct BenchShield Trajectories, a human-labeled corpus of 456 adjudicated trajectories from more than 31,000 public agent runs across three benchmarks. Compared with an agentic hackability scanner baseline on the same tasks and model, BenchShield improves full-chain recall from 23-94% to 77-100%, same-vector coverage from 16-56% to 43-78%, and reduces per-task cost by up to 65%. Its runtime analysis achieves 96% accuracy in detecting reward hacking from infrastructure-side evidence.