GitSwarm: Decentralized Compounding Inference
Vedant Shah, Carole-Jean Wu, Remi Munos, Rob Fergus, Jakob Foerster, Jason Weston and colleagues at Meta Superintelligence Labs (with Mila, CMU and Princeton) propose compounding inference, where intermediate agent work persists in a shared Git repository so later agents can extend, combine or challenge it, and implement it as GitSwarm.
Ask this paper
Substrate. Homogeneous asynchronous workers share one branchable Git repository. Each contribution is an atomic commit, and an `informed_by=` field records which earlier commits on other branches it depends on. No central planner assigns roles; workers decide whether to explore, verify, refine or combine.
Problem solving. With Codex (GPT-5.5-High) workers, GitSwarm solves all 30 IMOProofBench-Advanced problems in one run using under one million output tokens. On ProgramBench it reaches a 79.4% mean score against 65.1% for a single Codex agent prompted to continue at a comparable budget.
Scaling with workers. ProgramBench score rises from 73.1% with 100 workers to 79.4% with 500, while the sequential Codex baseline stays between 63.1% and 65.1% across its checkpoints.
Reuse is measurable. Later workers build on 94.7% of published ProgramBench contributions, and the selected solution's ancestry covers 81.6% to 92.6% of the contribution graph. In architecture research runs, 70.1% of commits labelled negative are cited by later accepted work.
Research setting. On three GPU-backed architecture tasks (Residual Matrix Transformer, Looped Transformer, NanoChat), workers ran 395 experiments in the first 26 hours across twelve idea families and improved on the starting architectures.
Abstract
Long-horizon problem solving and scientific research require computation to accumulate across successive attempts. Partial solutions, experimental findings, and unsuccessful approaches can inform later work, yet most inference-time computation is organized around individual trajectories or candidates rather than a persistent body of reusable work. We call this paradigm compounding inference: organizing inference-time computation so that intermediate work persists and can be inspected, extended, combined, or challenged by subsequent computation. We instantiate compounding inference in GitSwarm, an asynchronous system where homogeneous agents independently decide how to advance a task while collaborating through structured persistent memory. Agents explore, experiment, verify, refine, and synthesize previous work in a shared, branch-able Git repository. Atomic commits preserve intermediate artifacts, while explicit semantic dependencies record how later contributions build on work across branches. We evaluate GitSwarm on long-horizon problem solving and sustained GPU-backed experimental research. On IMOProofBench-Advanced, GitSwarm solves all 30 problems in one run using GPT-5.5. On ProgramBench, it achieves a $79.4\%$ mean score, versus $65.1\%$ for the strongest reported baseline under the stated budget. On three neural architecture research tasks (Residual Matrix Transformer, Looped Transformer, NanoChat), GitSwarm improves upon the starting architectures through successive experimentation. Beyond final performance, we measure whether computation accumulates: on ProgramBench, $94.7\%$ of contributions are subsequently built upon, while the selected solution's ancestry covers $82-93\%$ of the contribution graph. These results show that inference-time computation can accumulate across otherwise independent episodes, forming an evolving body of work that subsequent inference can reuse.