When Evaluators of AI Agents Disagree: An Empirical Study of Native Scoring, LLM Judges, and Commit-Boundary Checks on tau2-bench
Sungbin Im at aSSIST Seoul (now AIM Intelligence) scores 2,160 tau2-bench episodes with four evaluators and three-axis human labels, and finds that native scoring, LLM judges and a minimal rule-based check are not interchangeable.
Ask this paper
Setup. Airline and retail episodes are scored by native gold-state comparison, an LLM judge, the judge with the policy document, and a boundary evaluator that only checks whether a database write was expected against whether one happened.
Judge consistency is not accuracy. The judge repeated its own verdict 98.2% of the time but agreed with native scoring on only 56.2%.
Forbidden writes. In 11 episodes the agent executed a write the policy forbade. The gpt-4.1-tier judge passed 10 to 11 of them with or without the policy or transcript, and a stronger judge still passed 6. The boundary evaluator, with no model call, caught all 11.
Gates. Forbidden writes occurred in 11.1% of episodes without gates, and verification, correction and safe-termination gates did not reduce them significantly. Trace analysis showed the gates' evidence digest dropped order status and customer authentication.
Takeaway. The evaluator choice decides the conclusion, and a cheap commit-boundary check catches failures LLM judges miss.
Abstract
Agentic workflows, in which a large language model (LLM) calls tools to read and write data, are evaluated in two ways. Benchmarks compare the final state against a gold answer. Deployed services have none; an LLM judge reads the transcript and decides. Practice treats the two as interchangeable. This paper measures whether they are. We ran 2,160 episodes in tau2-bench's airline and retail domains and recorded every database write. We scored each episode four ways: native scoring against gold state, an LLM judge, that judge given the policy document, and the boundary evaluator, a minimal rule-based check of whether a write was expected against whether one occurred in the trace. Two annotators independently labeled 120 episodes for task correctness, policy compliance, and commitment correctness. The judge repeated its verdict 98.2% of the time but agreed with native scoring on 56.2%. In 11 episodes the agent executed a forbidden write. The judge passed 10 to 11, with or without the policy or transcript. A stronger judge still passed 6. The boundary evaluator, with no model call, caught all 11; no judge significantly exceeded it on human labels. Forbidden writes occurred in 11.1% of episodes without gates. Verification, correction, and safe-termination gates did not reduce them significantly. The traces show a candidate cause: the gates' evidence digest dropped the order status and customer authentication, so valid requests were rejected. The choice of evaluator decides the conclusion; the minimal check exposes failures the judges miss. Traces, code, prompts, and labels are released.