Compiling Agentic Workflows into Weights
Free while signed in. Answers cite the passages they came from.
This paper shows that a full agentic workflow can be distilled into the weights of a small model and run at roughly two orders of magnitude lower inference cost while preserving near-frontier task quality. Instead of keeping an external orchestrator above the LLM, the procedure is compiled into the weights of a fine-tuned model, producing what the authors call a subterranean agent.
The whole workflow, not just the answer: The compiled procedure includes multi-step LLM calls, tool invocations, intermediate scratchpads, and decision points. The student internalizes the orchestration logic rather than only imitating final outputs.
Orchestrator dissolved into the model: Classic agent frameworks run a planner loop above the model on every request. Compiling that loop into weights removes the per-call orchestration overhead, which is where most of the cost and latency live.
Near-frontier quality at 100x less cost: Across the evaluated tasks, the distilled small model stays close to the original workflow's quality while cutting inference cost by about two orders of magnitude. The savings come from collapsing many model calls into one forward pass.
Why it matters: Most production agents pay repeatedly for an orchestration loop they run thousands of times a day. If that loop can be compiled once into a cheap model, the economics of deploying agentic systems change substantially, especially for high-volume narrow workflows.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack