Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning
Xujia Wang, Juanzi Li and colleagues at Tsinghua University, with Shaohan Huang and Furu Wei of Microsoft Research, introduce WebCatalyst, a reinforcement learning framework that trains models to generate interactive HTML by playing the generated pages in a browser and scoring them.
Ask this paper
Testbed. Mini-games expose failures that static webpage tasks hide: pages that render but break during sustained interaction.
Game-level GUI assessor. An agent plays each generated page through mouse and keyboard to collect interaction evidence for the reward.
Visual Anchor. Stops the policy from exploiting the VLM reward with degraded visuals, which the authors identify as a form of reward hacking.
Action-Trace Credit. Assigns token-level credit using runtime execution traces, linking code tokens to the behaviors they caused.
Results. Trained on fewer than 1K requests, mostly games, Qwen3.6-35B-A3B rises from 68.8 to 75.3 on Cookie-Bench Mini Games and from 48.8 to 56.9 on WebGen-Bench, and ranks above Claude Opus 4.7 on average across metrics.
Abstract
Generating HTML from natural-language specifications is an important application of code large language models (LLMs). Despite producing renderable pages, current models often fail to ensure functionality and visual quality during actual interaction. Existing methods typically target task-specific webpage supervision, leaving demanding interactive artifact optimization underexplored. Using mini-games as a challenging testbed, we expose limitations in interaction experience, reward signal, and credit assignment fidelity that are obscured by shallow webpage interactions. We introduce WebCatalyst, a browser-interaction reinforcement learning framework with three complementary components: a game-level GUI assessor for sustained interaction and evaluation, Visual Anchor for mitigating VLM reward hacking, and Action-Trace Credit for grounding token-level credit assignment in runtime execution traces. Trained on fewer than 1K requests, predominantly for games, WebCatalyst raises the Cookie-Bench Mini Games score from 68.8 to 75.3 and also the WebGen-Bench accuracy from 48.8 to 56.9 on Qwen3.6-35B-A3B, outperforming Claude Opus 4.7 in average rank across evaluated metrics. Further analysis shows that Visual Anchor prevents aesthetic degradation, while runtime-grounded credit assignment accelerates functional improvement. Together, our results show that improving feedback fidelity is the key to substantially advancing interactive HTML generation.