Designing Agentic AI Workflow Portfolios under Imperfect Selection and Compute Cost
Mojtaba Abdolmaleki, Stefanus Jasin and Boyu Wang formulate the choice of how many agentic workflow runs to execute, and of which types, as a portfolio problem that trades extra correct candidates against compute and selection errors.
Ask this paper
Model. Selector quality is summarized by an odds-lift index, with sharp bounds on the value of workflow variety.
Algorithms. Exact formulations, LP relaxations, randomized rounding and certificates handle finite pools; a dual with an ellipsoid method and pricing oracle handles large workflow classes.
Results. Portfolio optimization improves held-out selector accuracy by 3.1, 7.5 and 0.9 points on ABCD, Schema-Guided Dialogue and HotpotQA; dual-guided workflow generation adds 3.5 points on ABCD and 24.1 on HotpotQA.
Abstract
Agentic AI systems often approach the same task through multiple workflows that differ in reasoning strategy, verification structure, and compute cost. A natural deployment policy is to use the workflow with the highest average performance, but this can be suboptimal because different workflows may succeed on different instances. We study a portfolio-and-selector paradigm in which a firm runs multiple workflow executions and selects the final answer after observing their outputs. Additional executions may uncover correct answers that the best standalone workflow misses, but they consume compute and introduce plausible distractors that complicate final selection. We formulate this as a workflow portfolio problem in which the firm jointly chooses run size and allocation across workflow types. We summarize selector quality through an odds-lift index and derive sharp bounds on the value of workflow variety. For finite workflow pools, we develop exact formulations, linear programming relaxations, randomized rounding procedures, and computable performance certificates. For large implicit workflow classes, we derive a finite-dimensional dual and an ellipsoid method using a pricing oracle to identify workflows with high weighted accuracy net of recurring compute cost. Under a weak condition, the method obtains a near-optimal solution to the relaxation with polynomially many oracle calls. We evaluate the framework on three datasets: ABCD, Schema-Guided Dialogue, and HotpotQA. Relative to the best standalone workflow, portfolio optimization improves held-out selector accuracy by 3.1, 7.5, and 0.9 percentage points, respectively. Dual-guided workflow generation adds 3.5 points on ABCD and 24.1 on HotpotQA, with no additional gain on Schema-Guided Dialogue.