ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
Siwei Wu, Chenghua Lin and colleagues at Beihang University, the University of Manchester and collaborating institutions propose ModularRSI, a framework for evolving agent harnesses that transfer to unseen tasks rather than overfitting the benchmark used during evolution.
Ask this paper
Three problems: Evolving on evaluation benchmarks mixes reusable improvements with benchmark-specific adaptation, single-trajectory updates confuse harness defects with instance details, and monolithic harnesses make changes hard to attribute.
Contrastive evidence: ModularRSI compares successful and failed trajectories of the same task and aggregates evidence across tasks to find recurring deficiencies.
Five modules: The harness is split into agent loop, tool use, observation management, context management and task completion detection, each evolved within a restricted scope and then integrated.
Results: Evolving on 2,000 executable tasks disjoint from the evaluation sets gives consistent gains on unseen in-domain and cross-domain tasks on TB2.0 and SWE-Bench Verified, and the evolved harness transfers across foundation models.
Abstract
Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve execution mechanisms from experience. However, generalizable harness RSI remains challenging. First, evolving harnesses on evaluation benchmarks or their subsets makes it difficult to distinguish reusable improvements from benchmark-specific adaptation. Second, single-trajectory updates can conflate systematic harness deficiencies with instance-specific reasoning and solution details, producing modifications that transfer poorly to unseen tasks. Third, localizing recurring behavioral deficiencies within monolithic harnesses is difficult, while whole-harness optimization can entangle unrelated mechanisms and complicate attribution and validation. We propose ModularRSI, a benchmark-disjoint, contrastive, and modular framework for generalizable harness evolution. ModularRSI contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies. It decomposes the evolvable harness into five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection. Each module evolves independently within a restricted modification scope, followed by an integration stage that combines the evolved modules into a unified harness and resolves potential conflicts. To support benchmark-disjoint evolution, we curate 2,000 executable evolution tasks from external sources that are disjoint from downstream evaluation benchmarks. Experiments on TB2.0 and SWE-Bench Verified show consistent improvements on unseen in-domain and cross-domain tasks, with the evolved harness also transferring across different foundation models.