Learning to Steer LLM Agents and Multi-Agent Systems
Leyang Shen, Fuli Feng, Tat-Seng Chua and colleagues (National University of Singapore, USTC and Alibaba's Qwen business unit) train a steering agent that watches a running agent or multi-agent system and intervenes when it drifts from the overall goal.
Ask this paper
Goal drift. Long-horizon tool use and inter-agent collaboration tend to move away from the original goal as execution continues, which wastes compute and produces unwanted outcomes.
Two paths. The steering agent uses a cheap fast path to decide when to intervene and an expensive slow path to decide how to intervene.
Training. Both paths are trained jointly with RL, combining a pairwise ranking objective with GRPO.
Results. On information-seeking search tasks, the steering agent improves both performance and efficiency of single-agent and multi-agent systems.
Generalization. The learned steering transfers to out-of-domain tasks and unseen executor models, and intervention frequency can be adjusted without retraining.
Abstract
Large language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these systems are prone to deviating from the overall goal as execution unfolds, which wastes execution resources and leads to undesired outcomes, a failure termed goal drift. In this work, we propose a steering agent that supervises execution on the fly and steers the system back on track when drift occurs. The steering agent observes the execution context, deciding when to intervene with a low-cost fast path and how to intervene with a high-cost slow path. To unlock its steering intelligence, the ability to assess goal drift and generate effective interventions, we jointly train the two paths with reinforcement learning, combining a pairwise ranking objective with group-relative policy optimization (GRPO). Experiments on information-seeking search tasks show that the steering agent provides effective supervision for both single-agent and multi-agent systems, improving their performance and efficiency. The learned steering intelligence generalizes to out-of-domain tasks, transfers to unseen executors, and supports controllable intervention frequency without retraining.