Elastic Horizon: Discovering the Effective Interaction Frontier in Agentic Reinforcement Learning
Gangyi Zhang in the Qwen Business Unit of Alibaba with USTC collaborators propose the effective interaction frontier hypothesis and Elastic Horizon, a closed-loop controller that sets an agent's interaction budget from the 90th percentile of successful trajectory lengths instead of a hand-set maximum.
Ask this paper
Existing curricula are open-loop: Horizon schedules increase monotonically to a manually specified maximum with no mechanism to detect when further expansion stops helping, so cost keeps growing after returns flatten.
The frontier hypothesis: There is a dynamic boundary past which additional environment interactions give diminishing returns while cost grows linearly, and fixed-horizon sweeps on AppWorld and BFCL show clear saturation plateaus consistent with it.
The controller is a percentile: Elastic Horizon tracks the boundary using the 90th percentile of successful trajectory lengths, which is a cheap statistic already available during training.
It converges from both sides: The controller stabilizes the horizon inside the saturation band starting from both under-capacity and over-capacity initializations, so it does not depend on a good initial guess.
Best success rates plus 25% token savings: It attains the best success rates across 7B and 14B backbones while saving up to 25% of per-step trajectory tokens, reframing the question from how to scale horizons to when to stop.
Abstract
Scaling the interaction horizon-the maximum number of environment interactions per episode-improves LLM agents on long-horizon tasks, and curriculum-based methods that progressively expand the horizon outperform fixed-horizon alternatives. However, existing schedules are open-loop: they monotonically increase the horizon until a manually specified maximum, with no mechanism to detect when further expansion stops helping. We propose the effective interaction frontier hypothesis: a dynamic boundary beyond which additional interactions yield diminishing returns while cost grows linearly. We then introduce Elastic Horizon, a closed-loop controller that tracks this boundary via the 90th percentile of successful trajectory lengths. On AppWorld and BFCL, fixed-horizon sweeps reveal clear saturation plateaus; Elastic Horizon stabilizes the horizon inside the saturation band from both under- and over-capacity initializations, attains the best success rates across 7B and 14B backbones, and saves up to 25% of per-step trajectory tokens. Our work shifts the paradigm from how to scale interaction horizons to when to stop scaling.