System Optimizations for Cost-Efficient Agent Harnesses
Jongseok Park, Ion Stoica, Alvin Cheung and colleagues at UC Berkeley study where agent harnesses spend tokens and build Agency, a platform that applies cost optimizations to existing harnesses such as Claude Code and Codex without changing the model.
Ask this paper
Observation. Harness choices such as system prompts and tools change task accuracy little but change cost by up to 2.6x on the same model and task.
Analysis. Analysis of real and benchmark traces identifies five opportunities to reduce how much context each LLM call reads and how many calls an execution makes.
Agency. A harness interposition platform with one abstraction over off-the-shelf harnesses, so each system optimization is written once and works across harnesses and models.
Result. Five proof-of-concept techniques that compress context or remove tool calls cut cost by up to 35% with marginal accuracy loss.
Abstract
AI agent harnesses play the critical role of providing LLMs with an interface to a computing environment. Recent studies show that AI agents' task-execution accuracy depends mostly on the LLM used, while harness-side variances such as system prompts or tools have minimal impact. Rather, such harness-side variances show a larger impact on the cost of agent executions, achieving cost reductions up to 2.6 times on the same model and task. To further explore these harness-side cost optimization opportunities, we analyze the cost accumulation patterns across various agent traces and identify system opportunities to reduce token use and LLM calls without modifying the model or affecting task accuracy. Based on this analysis, we design Agency, a harness interposition platform that provides a unified abstraction and allows easy and portable implementation of harness-side system optimizations on existing off-the-shelf harnesses such as Claude Code or OpenAI Codex. Using Agency, we implement proof-of-concept optimization techniques based on our analysis and achieve cost reductions up to 35% across various real-world benchmarks on existing harnesses without compromising accuracy.