AgentFactory: Towards Automated Agentic System Design and Optimization
Enci Zhang and colleagues at Peking University present AgentFactory, which jointly optimizes the foundation model and the workflow structure of an agentic system under performance, cost, and efficiency objectives rather than optimizing the workflow alone.
Ask this paper
The gap it targets. Prior automated workflow search treats the model as fixed and optimizes a single metric, which ignores that model choice and workflow shape trade against each other under deployment constraints.
LLMs as the optimizer. A three-stage pipeline uses advanced models to navigate the configuration space and discover combinations of fine-tuned models and optimized workflows.
Broad evaluation. Eight benchmarks across general reasoning, coding, mathematics, medicine, and finance, with an average 9.1% improvement over both manually designed and existing automated approaches.
Domain-specific gains are the standout. 19.6% on MedQA and 18.7% on FinEval, which suggests joint optimization pays off most where a specialized model plus a matched workflow beats a generic strong model.
Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities as powerful components in agentic systems, enabling sophisticated reasoning and complex task execution. However, current approaches to manually designing and optimizing agentic systems heavily rely on manual effort, limiting their adaptability and scalability. Recent work has explored the automated optimization of workflow designs. However, these approaches often overlook the crucial role of model capabilities and focus on single performance metrics, failing to address real-world deployment constraints. In this paper, we present AgentFactory, a framework that jointly optimizes both foundation models and workflow structures in agentic systems while considering multiple objectives including performance, cost, and efficiency. AgentFactory leverages advanced LLMs as optimizers to navigate the vast search space of possible configurations, employing a three-stage optimization pipeline to automatically discover effective combinations of fine-tuned models and optimized workflows. Through an iterative optimization process, our framework systematically explores and evaluates different agentic system designs, adapting to task-specific requirements while maintaining operational efficiency. We evaluate AgentFactory across eight benchmarks spanning five domains, including general reasoning, coding, mathematics, medicine, and finance. Our experiments demonstrate that AgentFactory consistently outperforms both manually designed methods and existing automated approaches, achieving an average improvement of 9.1% across all benchmarks, with particularly significant gains in domain-specific tasks (19.6% on MedQA and 18.7% on FinEval). These results establish AgentFactory as a promising approach for developing more capable and efficient agentic systems through automated optimization.