🚀NEW LABGetting Started with Claude AgentsStart lab
← All papers  /  Oct 21, 2026
Agents

Does Multi-Agent Collaboration Pay When Agents Serve Users? Testing Agent Scaling in Dual-Control Settings

First page
Does Multi-Agent Collaboration Pay When Agents Serve Users? Testing Agent Scaling in Dual-Control Settings
The curator’s take

Ethan Yang and Xiangcheng Shi (University of Pennsylvania) test, in a pre-registered study, whether a multi-agent system beats a single agent when agents serve a user under a written policy, using the dual-control tau2-bench.

Ask this paper

Key points
01

Design. The stock single agent is compared with a centralized MAS built from the same model on the airline, retail and telecom domains, for six models from three families over about 7,950 simulated conversations, with a completion-token ledger per episode per system.

02

Result. Under a fixed simulated user, the MAS is worse in 12 of 13 valid cells and tied in the thirteenth, with a weighted mean of -28 points (95% CI [-44, -12]).

03

Prior rule fails. On the two cells where the 45% capability-saturation rule predicts a benefit, the MAS loses by 13 and 22 points, and without a budget ceiling it at best ties while spending 5.5 times the tokens.

04

Failure shift. The MAS does not detectably reduce wrongful database writes. It converts other failures into budget exhaustion and inaction.

05

Disclosed limit. The authors report one unmet validity bar: the MAS planner never saw the tool schemas.

Abstract

Multi-agent systems (MAS) built from language models are deployed on the premise that collaboration beats a single-agent system (SAS), yet when it pays is unclear. The most systematic study to date gives a capability-saturation rule: coordination helps while the SAS is weak and hurts once it is strong, with the crossing near 45% single-agent accuracy. That rule was derived on autonomous benchmarks, where no user takes part. We test it where agents now work: in conversation with a user under a written policy, on $τ^2$-bench, the dual-control benchmark. On its airline, retail and telecom domains we compare the stock SAS with a centralized MAS built from the same model for six models from three families, over about 7,950 simulated conversations, with one completion-token ledger per episode per system. The study was pre-registered with a public change log; the MAS is reported as built, with one validity bar unmet (its planner never saw the tool schemas). Under a fixed simulated user no cell shows a MAS benefit: the MAS is worse in 12 of the 13 valid cells and tied in the thirteenth, and the fitted gain is negative across the observed capability range (weighted mean -28 points, 95% CI [-44, -12]). On the two cells where the rule predicts a benefit, the MAS loses by 13 and 22 points, and without a budget ceiling it at best draws level at 5.5 times the SAS's token spend. The one gain appears on the cell nearest 45%, under a second simulator. Whether the gain declines with capability is unresolved (slope -0.41, 95% CI [-1.51, +0.68]), because model-by-domain effects dominate. The MAS does not detectably reduce wrongful database writes; it converts other failures into budget exhaustion and inaction. For agents that serve people, the SAS is the default; a multi-agent design must show compute-matched evidence from the setting itself. We release all trajectories, ledgers, prompts and code.

Every Monday
Get next week’s papers.
Subscribe on Substack