ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
Zhuoshi Pan and colleagues at Tencent and Tsinghua train an agent to manage its own working context with a purpose-built RL method, adding planning, long-term memory, and soft offloading tools and assigning credit at the level of individual context edits (EMNLP 2026 main).
Ask this paper
The gap they name: Existing proactive context management gives models only search, delete, and summarize, explores context actions as if they were interchangeable, and then broadcasts one trajectory-level reward across every edit.
Expanded toolset: ContextPilot adds global planning, long-term memory, and adaptive soft compression, so the agent can offload rather than only discard.
Branch sampling on critical edits: Context and entropy variation identify which editing decisions actually matter, and branches are sampled there; action-level advantages are then estimated from all branched trajectories passing through that edit.
Result shape: Stronger long-context QA and deep-search performance with a smaller working context, consistently across base models and benchmarks, which is the combination that matters for cost.
Open code: Released at github.com/Tencent/ContextPilot, so the credit-assignment recipe is reproducible.
Abstract
Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, a proactive context management framework for long-horizon agentic reasoning. Our approach systematically augments the toolset with planning, long-term memory, and soft context offloading tools. We further propose an RL method tailored for context management, which uses context and entropy variation to identify critical editing decisions for branch sampling and estimates action-level advantages from all branched trajectories that pass through the corresponding context editing action. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming existing baselines across various base models and benchmarks. Code is available at https://github.com/Tencent/ContextPilot.