Agent-as-a-Router
Free while signed in. Answers cite the passages they came from.
Most users now have access to many LLMs that each excel in different domains, so routing each task to the right model matters for both quality and cost. Existing routers treat this as a static, one-off classification problem, and this paper shows that framing is exactly what holds them back.
Information deficit is the bottleneck: Simply augmenting a vanilla LLM router with performance statistics at the task-dimension level yields a 15.3% relative gain, surpassing a heuristic router built on the same priors, which pinpoints missing information rather than model choice as the real limiter.
Routing as a closed loop: Agent-as-a-Router formalizes routing as a Context, Action, Feedback, Context loop that accumulates execution-grounded experience during deployment instead of deciding once and moving on.
A concrete system and benchmark: The framework is instantiated as ACRouter, built from an Orchestrator, a Verifier, and a Memory module, and the authors release CodeRouterBench, roughly 10K task instances scored across 8 frontier LLMs for regret-based comparison on streaming tasks.
Why it matters: ACRouter achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic programming, showing that treating routing as an experience-gathering agent, not a classifier, is what closes the information gap.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack