ForkPilot: Self-Evolving Policy for Retrospective Search in Long-Horizon Agents
Xinyue Zeng (Virginia Tech, Microsoft), Shivam Shandilya, Ranveer Chandra, Emre Kiciman, Tusher Chakraborty and colleagues at Microsoft propose ForkPilot, a learned policy that decides when a long-horizon agent should fork back to an earlier state and search again.
Ask this paper
Problem. Retrospective search recovers from compounding errors but wastes tokens when it forks at the wrong time. The authors name two causes: delayed outcomes make it hard to credit individual search decisions, and the value of searching changes as execution evidence accumulates.
Method. Stage one learns a search-value policy offline from completed trajectories using automatically built outcome comparisons. Stage two makes fork decisions online and keeps updating the policy with newly completed trajectories.
Results. Averaged over benchmarks, ForkPilot cuts raw inference tokens by 36.2% and uncached tokens by 30.3% against full retrospective search, while task score changes by +0.23 points. Savings reach 59.2% on AutomationBench and tau3-Bench.
Scale and deployment. Token savings grow with backbone size from 28.5% at 4B to 37.0% at 27B. The evaluation covers seven model families, including GPT-5.6 Sol and Opus 4.8 inside a production harness used by hundreds of thousands of paid users.
Data efficiency. With 25% of the training split, ForkPilot already captures 81% to 85% of the gain from the full split.
Abstract
Interactive language-model agents increasingly solve complex tasks through long-horizon, multi-call reasoning, where errors in beliefs or actions can compound across tool interactions. Retrospective search can recover from such failures but is prone to misallocation. Delayed outcomes obscure the contribution of intermediate search decisions, leading to Attribution Complexity, while evolving execution evidence leads to Adaptation Complexity, where previously learned estimates become stale. To address these challenges, we first introduce Search Value Dynamics (SVD), which characterizes the evolving trade-off between the gain and cost of retrospective search. Building on SVD, we propose ForkPilot, a self-evolving two-stage policy-learning framework. In the first stage, ForkPilot learns a search-value policy offline from completed trajectories through automatically constructed outcome comparisons. In the second stage, it makes search decisions based on current observations and then self-evolves by incorporating newly completed trajectories into subsequent policy updates. We evaluate ForkPilot across 6 diverse benchmarks and 7 widely used LLM backbone families, including four open-source families, GPT-5.6 Sol, and Opus 4.8 in a production agentic system, against 9 competitive baselines, including a real-world harness deployment used by hundreds of thousands of paid users. ForkPilot achieves comparable state-of-the-art performance while reducing token usage by up to 59.2%, demonstrating its efficacy.