LongCapHarness: Capability-Guided Auto-Harnessing for Long-Context Reasoning
Tao Zhang and Yuhang He (Microsoft Research Asia) improve long-context reasoning by building a harness with recursive self-improvement instead of synthesizing data and post-training the model.
Ask this paper
Approach. LongCapHarness decomposes long-context tasks into capability primitives. A harnessing agent revises the primitive implementations, the capability state and the control logic through recursive self-improvement on a development set.
No training. No data synthesis and no post-training of the target model. The harness is built for Llama3-8B and transferred to Qwen2.5-3B.
Results. On HELMET and RULER it outperforms recent data-synthesis and training baselines. With Llama3-8B it beats the base model by 64.18 points on average and also beats NExtLong and the RRSI self-improvement baseline.
Cost. Estimated development cost is under 50% of the estimated training cost of NExtLong and PolicyLong.
Limitation. The harness is built by a strong agent (GPT-5.6-Sol). Performance drops sharply when Llama3-8B or Qwen2.5-3B build their own harnesses.
Abstract
Existing methods enhancing an LLM's long-context reasoning capability often rely on data synthesis and follow-up LLM post-training. However, the data synthesis strategy and the post-training recipes are empirically driven and computationally intensive. In this work, we propose a new framework, dubbed LongCapHarness, to instead harness the underlying capabilities for long-context reasoning. The harnessing process recursively revises and updates capability primitive implementation, capability state and control logic through recursive self-improvement on the development set. With neither data synthesis nor LLM post-training, LongCapHarness is applied to Llama3-8B and transferred to Qwen2.5-3B. LongCapHarness substantially outperforms recent data-synthesis-based baselines on HELMET and RULER benchmarks; its estimated development cost is less than 50% of the estimated training costs of NExtLong and PolicyLong. We hope these results motivate further research into harnessing the latent capabilities of LLMs to solve tasks beyond their explicit training.