ReproBridge: Building Reproducibility into ML Research Artifacts
Yubo Wang (Waterloo, Google intern) with Rui Meng, Jaehyun Nam, Tomas Pfister and Jinsung Yoon (Google) test whether saving the procedures an agent recovers while reproducing an ML paper makes the next independent reproduction more reliable, and build ReproBridge to produce those saved artifacts.
Ask this paper
Artifact design. Each refined artifact has four parts: a cleaned codebase, environment setup, experiment launchers and a reproduction guide. An agent workflow (planner, builder, reproducer, distiller, packager, reviewer) constructs them.
Benchmark. ReproBridge-Bench has 1,029 tasks from 387 experiments across 36 papers. Each task is anchored by five human-verified reference scores, and a separate Auditor checks execution evidence.
Results. Across four models, refined artifacts raise mean reproduction success from 76.4% to 92.6% and cut downstream API spending by 19.7%. All four models exceed 90% mean success.
Model gap. The spread between the best and worst model shrinks from 21.6 to 3.9 points, so weaker reproducers benefit most.
Ablation. Removing any of the four components lowers success. Removing code or environment setup causes the largest losses.
Abstract
Reproducing machine learning experiments requires reconstructing procedures from information scattered across papers, released code, and supporting artifacts. For coding agents, reproduction success can therefore depend on their ability to fill in missing details, rather than on the released artifacts alone. We ask whether preserving recovered procedures and decisions improves subsequent independent reproduction. We define a standardized artifact design comprising a refined codebase, environment setup, experiment launchers, and a reproduction guide, and introduce ReproBridge, an agent workflow for constructing these refined reproducible artifacts. ReproBridge-Bench provides 1,029 tasks from 387 experiments across 36 papers, anchored by five human-verified reference scores per task. Agents attempt tasks in fresh sessions with raw code or refined artifacts; a separate Auditor checks execution evidence. Across four evaluated models, refined artifacts raise mean reproduction success from 76.4% to 92.6% and reduce downstream API spending by 19.7% on average. The cross-model success-rate spread narrows from 21.6 to 3.9 percentage points, with all four models achieving mean success rates above 90%. Removing any component reduces success in the ablation study, with the largest losses from code and environment removal. These results suggest that preserving execution knowledge helps even models with lower raw-code success reproduce experiments more reliably and at lower API cost.