Compiled Agency: Frontier General-Purpose Coding Agents Build Winning Game Players from Bare Interaction - from Flappy Bird to StarCraft II and Civilization
Joey Xiao (New York University) and Haonan Huang (Princeton University) introduce Gauntlet, a develop-freeze-evaluate framework in which a general-purpose coding agent receives only a game description, a raw observation and action interface and an empty policy file, then writes a standalone controller that is scored with no model calls during play.
Ask this paper
Bare contract: The agent gets no strategy, algorithm or architecture; in one autonomous session it experiments with the live game and writes the player, which is then frozen.
Generational threshold: On an unpublished procedural roguelike, held-out success ranges from 0 to 86%, and every observed session of the newest model generation beats the best session of its predecessor.
StarCraft II: A compiled raw-API controller defeats every fair built-in StarCraft II AI and two cheating variants on held-out map, opponent and seed scenarios.
Civilization: Single-session programs win complete Freeciv games by conquest on held-out seeds, at modest rates against novice AI, which the authors report as the first standalone language-agent wins in the genre.
Compiled agency: The resulting controllers are inspectable programs, so the strategy the model developed can be read directly rather than inferred from per-turn calls.
Abstract
LLM agents have repeatedly struggled to convert knowledge of a game into competent play, even when researchers build the agent around the model - supplying perception, memory, skill libraries, planners, or executable-policy scaffolds. Rapid progress in coding agents raises two sharper questions: can frontier models now win games at all, and can they win them unaided, building the entire player themselves? We introduce Gauntlet, a develop-freeze-evaluate framework that ports games from small arcades to full commercial-scale titles, behind one deliberately bare contract: a general-purpose coding agent receives a game description, a raw observation/action interface, and an empty policy file - no strategy, no algorithm, no architecture. In a single autonomous session the agent experiments with the live game and engineers a standalone controller; we freeze the result and score it on held-out instances with zero model calls during play. On an unpublished procedural roguelike, held-out success spans 0-86 percent and exposes a sharp generational threshold: every observed session of a newest-generation system outperforms the best session of its predecessor. At full-game scale, a compiled raw-API controller defeats every fair StarCraft II built-in AI and two cheating variants, and single-session programs win complete games of Civilization (Freeciv) by total conquest on held-out seeds. Though at modest rates against novice AI, this is a first: no prior language-agent system had won full games of this genre standalone, without per-turn model calls and a hand-crafted tactical layer. Frontier coding agents begin to track long-horizon strategy. The frozen programs are inspectable. We call this capability compiled agency: development experience compiled into a persistent executable agent whose architecture is built by the model.