GenXR-Bench: Benchmarking Coding Agents for Interactive XR Development
William Huang (UCLA), David Li and colleagues at Google XR Labs with UCLA, CMU, MIT and others introduce GenXR-Bench, the first end-to-end benchmark for coding agents building interactive extended-reality applications.
Ask this paper
Tasks. 112 expert-authored tasks: 24 feature implementations, 21 bug repairs and 67 full application generations.
Tests that move. Held-out tests combine scripted and adaptive interactions with programmatic and visual assertions, and agree with blinded expert judgments on 124 apps run on consumer XR hardware 96.5% of the time.
Headline number. Across 10 models, the best configuration reaches 50.9% overall Pass@1.
Failure modes. Agents repeatedly misunderstand asynchronous sensor lifecycles, physical grounding and the semantics of embodied interaction, which do not appear in ordinary repository benchmarks.
Abstract
Coding agents are rapidly advancing from single-file generation to long-horizon repository-scale engineering, yet the evaluation of their capabilities to author interactive extended reality (XR) applications remains an open challenge. Unlike traditional software, XR applications operate within dynamic 3D physical environments where runtime robustness depends jointly on continuous user motion, spatial input, and environmental sensing. We introduce GenXR-Bench, the first benchmark for the end-to-end evaluation of coding agents in XR software development. GenXR-Bench contains 112 expert-authored tasks spanning 24 feature implementations, 21 bug repairs, and 67 application generation tasks. Held-out tests combine scripted and adaptive interactions with programmatic and visual assertions to evaluate core functionality and runtime behavior. GenXR-Bench's testing suite achieves 96.5% accuracy against blinded expert judgments of 124 candidate applications executed on consumer XR hardware. Evaluating 10 models on GenXR-Bench, we find that the strongest configuration achieves only 50.9% overall Pass@1. Failure analysis identifies recurring misunderstandings in asynchronous sensor lifecycles, physical grounding, and embodied interaction semantics. GenXR-Bench establishes a reproducible testbed for evaluating coding agents in the interactive and physically situated conditions of spatial computing.