When Is Routing Meaningful
Free while signed in. Answers cite the passages they came from.
LLM routers and mixture-of-agents systems get judged on accuracy and cost, both of which can look great while the router is doing nothing. This DeepMind-affiliated work argues that whether routing means anything depends on two properties that are orthogonal to accuracy.
Two conditions for real routing: The society of models must be behaviorally differentiated, since routing is vacuous when every actor responds the same way, and assignments must stay stable when a query is rewritten.
A diversity measure that sees structure: The authors use Hierarchic Social Entropy to score how genuinely different a pool of models is, showing purpose-built specialist societies are far more diverse than large real-world model pools of similar size.
Accuracy hides fragility: Learned KNN routers gain accuracy on specialist societies yet collapse under paraphrase perturbations, while a prompted router keeps both accuracy and robustness, so clean-query accuracy alone can mask a meaningless router.
Why it matters: These two checks catch routers that look good and do nothing, and they show that a small, carefully curated society can recover most of the diversity of a much larger pool.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack