Learning How Much to Collaborate: Difficulty-Aware Topology Selection for Multi-Agent Code Generation
Yunsong Hong (University of Sydney) shows that the benefit of hierarchical multi-agent code generation grows sharply with problem difficulty and proposes DATS, which picks a communication topology for each problem by trading predicted success against cost.
Ask this paper
Difficulty dependence. Across 614 problems from APPS, HumanEval+ and LiveCodeBench, hierarchical collaboration beats a single agent by 2.4 pass@1 points on the easiest third and 21.1 on the hardest third, at about ten times the token cost.
Selector. DATS predicts the probability that each of five topologies solves a problem and chooses the one with the best success minus cost, with a single scalar cost penalty that can be recalibrated without retraining.
Budget-matched results. At 40% of the always-hierarchical cost, DATS reaches 77.7% pass@1 against 73.6% for always-hierarchical and 74.3% for the strongest learned competitor, holding across four backbones and on 400 math problems.
Abstract
Multi-agent systems for code generation are deployed with a single communication topology, chosen once for every problem. This is the wrong granularity. Evaluating five topologies on 614 problems from APPS, HumanEval+ and LiveCodeBench, we find that the advantage of hierarchical collaboration over a single agent grows from 2.4 points of pass@1 on the easiest third of problems to 21.1 points on the hardest third, while its token cost stays about ten times higher. We propose the Difficulty-Aware Topology Selector (DATS), which predicts each topology's probability of solving a problem and selects the one maximising predicted success minus cost. Its predictor is a graph network that treats the five topologies as nodes of a connectivity order rather than independent labels, worth 1.7 points over a flat multi-label head. Because the cost penalty is a single scalar recalibrable without retraining, routers compare at equal spend: under this budget-matched protocol six cost-aware methods span 21.6 percentage points, and two baselines leading DATS fall behind once calibrated to it. Fixed at 40% of the always-hierarchical cost, DATS reaches 77.7% pass@1 against 73.6% (always-hierarchical) and 74.3% (strongest learned competitor), all eleven pairwise McNemar comparisons surviving Holm-Bonferroni correction. The 4.1-point gain holds across four backbones spanning fourteen points of capability, and replacing the 39 interpretable features with a graph network or a pretrained encoder shifts accuracy by at most 1.3 points, never significantly. A cross-domain study on 400 mathematical reasoning problems reproduces the effect, the gap widening from 2.5 to 20.9 points.