Comparative Analysis of Agent Evaluation Frameworks: Stability, Detection, and Discovery in Enterprise Analytics Agents
Howard Zhang, Chuck Arvin and colleagues at Amazon compare three agent evaluation frameworks (Strands Evals, PromptFoo and Agenta) on two supply-chain analytics agents using frozen execution traces, and find that what the evaluator can see decides how good the evaluation is.
Ask this paper
Three paradigms, one setup. Trace-based LLM judging, text-only LLM judging with red-teaming, and deterministic heuristics are run on the same frozen traces, so differences come from the evaluator and not from agent variance.
Stability. 97% of (framework, model) configurations meet the stability threshold across repeated runs, and the stability pattern carries over from one agent to the other.
Detection. On 72 ground-truth cases, trace-based judging reaches F1 = 0.90, catching 91% of failures at a 10% false-positive rate. Text-only judging matches recall at 88% but raises 5x more false alarms because it cannot see tool outputs.
Discovery. Automated testing found 24 new failures: 9 functional (SQL bugs, tool workflow gaps) from trace-based evaluation and 15 security failures (schema leakage, storage path exposure, prompt disclosure) from red-teaming, with zero overlap between the two.
Human labels. The evaluators flagged 9 ground-truth failures that the human reviewer had mislabeled. The recommended setup is trace-based evaluation for functional checks plus adversarial red-teaming for security.
Abstract
Deploying LLM-based analytics agents in enterprise settings requires evaluation frameworks that can reliably detect failures across complex, multi-tool workflows. We present a three-phase comparative study of three evaluation frameworks, each representing a distinct evaluation paradigm (trace-based LLM judging, text-only LLM judging with red-teaming, and deterministic heuristics), applied to two analytics agents in a controlled research setting using frozen execution traces. Phase 1 quantifies evaluation harness stability through repeated measurements, finding that 97% of configurations meet the stability threshold and that stability patterns are agent-independent. Phase 2 measures known risk detection on 72 ground-truth test cases, where the trace-based framework achieves the best precision-recall balance (F1=0.90) while text-only evaluation matches on recall but generates 5x more false alarms. Phase 3 evaluates unknown risk discovery, uncovering 24 novel failures with zero overlap between functional testing and security red-teaming. Our results demonstrate that (1) the information available to the evaluation harness determines evaluation quality, (2) functional and security evaluation are complementary, and (3) LLM-based evaluation harnesses surface subtle errors that human reviewers miss. We recommend a two-framework strategy: trace-based evaluation for functional assessment and adversarial red-teaming for security validation.