Data World: Scaling Data Environments from Scratch
Zhanli Li and colleagues (Meituan Longcat Interaction Team with Hunan University, Nanjing University and others) introduce Data World, a multi-agent engine that first specifies the mechanisms behind a dataset and then generates the files, tasks and rubrics, producing verifiable environments for training and evaluating data agents.
Ask this paper
Core idea. Specifying the world that produced the files before generating the files gives each task a known ground truth, which solves the verification problem for open-ended analytical findings.
Audits and rubrics. Independent audits check that targets can be recovered from the observed files, and rubrics serve both evaluation and RL rewards.
Outputs. DataWorld-Bench has 100 human-curated tasks across 50 held-out worlds and 16 domains. The engine also produces 2,135 training tasks from 1,037 disjoint worlds.
Benchmark result. Ten frontier models find the benchmark hard, especially insight discovery, and show a consistent gap between computing correct numbers and drawing the right insight.
Training result. RL-zero training of Qwen3.6-35B-A3B on the synthetic tasks, without trajectory distillation, gains 7.06 points on DataWorld-Bench, 14.28 on DAComp, 20.90 on DABstep-Hard and 12.13 on JobBench.
Abstract
Data agents use code to process heterogeneous files and derive analytical insights. Scaling their training faces costly environment curation, privacy and intellectual-property restrictions, and difficulty verifying open-ended findings without known generative truth. Our core insight is to specify the world behind the files before generating the files themselves. We introduce Data World, a multi-agent engine that synthesizes environments, tasks, and rubrics from explicit mechanisms. Independent audits check target recoverability from observed files; rubrics support evaluation and RL rewards. Human preference-based curation for question realism selects 100 tasks across 50 held-out worlds and 16 domains for DataWorld-Bench. The engine also yields 2,135 training tasks from 1,037 disjoint worlds. Evaluation of ten frontier models shows that the benchmark remains challenging, particularly for insight discovery, with a consistent computation-insight gap. RL-zero training on the synthesized tasks produces DataWorld-35B-A3B from Qwen3.6-35B-A3B without trajectory distillation. Under matched evaluation, it improves over its base by 7.06 points on DataWorld-Bench, 14.28 on DAComp, 20.90 on DABstep-Hard, and 12.13 on JobBench, a broader occupational benchmark.