Where Should a Stuck Agent Spend Its Next Tokens? Causal Attribution of Inference-Time Resources
Saiyue Lyu, Haibo Zhang and colleagues (UBC, Toronto, Tulane, Rutgers) introduce Banzhaf Resource Attribution, a factorial method that measures how much extra computation, information and feedback each contribute when a failed coding agent tries again.
Ask this paper
Controlled setup. The failed state and revision protocol are held fixed while resources are toggled, across six models, 194 function-level failures (MBPP+, HumanEval+) and 126 SWE-bench Verified failures.
What helps. Task-relevant specifications consistently improve repair. Public-test feedback gives small gains, and retrieved code gives little or none.
Scaling behavior. The value of more computation falls as model capability rises, while information stays useful; most channels show diminishing returns, though complementary information can compound.
Agents pick the wrong resource. Three of four frontier models choose feedback in 95-100% of cases and gain less than uniform random resource selection.
Abstract
Inference-time scaling is often treated as a token-budget problem, but a failed coding agent can seek qualitatively different resources: computation, information, or feedback. We introduce Banzhaf Resource Attribution (BRA), a controlled factorial framework that measures each resource's marginal contribution to repair and its interactions while holding the failed state and revision protocol fixed. We evaluate six models on 194 function-level failures from MBPP+ and HumanEval+ and 126 repository-level failures from SWE-bench Verified. Task-relevant specifications consistently improve repair, whereas public-test feedback provides small gains and retrieved code little to none. The value of additional computation declines with model capability, while information remains useful; resource channels largely exhibit diminishing returns, although complementary information can reinforce itself. Yet three of four frontier models select feedback in 95-100% of cases and obtain lower gains than uniform random resource selection. BRA thus exposes a gap between the resources agents prefer and those that measurably improve repair.