The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents
Benjamin Gruenbaum, Doron Porat, Assaf Natanzon and colleagues at Eon generate a complete fictional company, including simulators of Salesforce, Zendesk, Slack and Gong, so enterprise agent answers can be graded exactly against computed answer keys.
Ask this paper
The evaluation problem: Enterprise agents cannot be evaluated on customer production data, and no existing substitute supplies ground truth.
One seeded entity graph feeds everything: Industry, company size, business model, application portfolio and a seed define each company. A single seeded entity graph supplies shared data to every product simulator, and a question-conditioned generator creates internal-database schemas and records from the same graph before adding database-specific facts, so the whole estate stays consistent.
Grading is exact: Every expected answer is computed from the final records rather than annotated, so there is no judge in the loop.
Realism is validated, not assumed: A realism scorecard and an adversarial detector validate the entity graph. Across 23 generated companies mean realism rose from 61.8 to 97.0, with zero records flagged as synthetic.
Model results: In the simulator-track comparison nine models answered the same 33 questions three times each, with accuracy from 42.4% to 76.8%. Three of 36 pairwise differences remained supported after correction, which the authors report rather than overstate.
Abstract
LLM agents for enterprise systems of record cannot be evaluated on customer production data, and no existing substitute provides ground truth. We present the Era by Eon Benchmark for evaluating LLM agents that use enterprise tools. The benchmark is built around a complete fictional company. It includes product simulators, company-specific internal databases, benchmark questions, and computed answer keys. Industry, company size, business model, application portfolio, and a seed define each company. One seeded entity graph supplies shared company data to simulators of Salesforce, Zendesk, Slack, Gong, and other products. A questionconditioned generator creates the schemas and records for internal databases. It takes shared entities, keys, and values from the same graph before generating database-specific facts. Both mechanisms therefore describe one consistent enterprise estate. Every expected answer is computed from the final records, so grading is exact. Design and answer-key checks validate the internal databases. A realism scorecard and adversarial detector validate the entity graph. Across 23 generated companies, the mean realism score rose from 61.8 to 97.0, with zero records flagged as synthetic. In the reported simulator-track comparison, nine models answered the same 33 questions three times each. Accuracy estimates ranged from 42.4% to 76.8%, and three of 36 pairwise differences remained supported after correction.