Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses
Weichen Yu, Tomas Pfister, Long T. Le and colleagues (Google Cloud AI Research, Google DeepMind, Carnegie Mellon University) present Harmony Harness, which evolves agent harnesses against several objectives at once and evolves general mechanisms separately from benchmark-specific adaptations.
Ask this paper
Single objectives trade off. Evolving for functionality alone lifts functional-and-secure pass on SecRepoBench from 46.7 to 56.7 but leaves security at 60.0. Evolving for safety alone adds 10 points of security and no additional secure passes.
Method. Harness evolution is cast as multi-objective optimization with decomposition-based (Tchebycheff) evolutionary search, which keeps candidates across the Pareto frontier. Factorized operators refine general harness mechanisms and benchmark-specific adaptations separately, and a dynamic workflow composes them at inference.
Results. On SecRepoBench the evolved harness reaches 76.7 functional-and-secure pass on the evolve split and 86.7 on held-out in-distribution tasks, against 50.0 for the baseline. On BaxBench it averages 61.3, against 42.9 for the baseline and 52.4 for the strongest meta-harness variant.
Ablation. Tchebycheff joint evolution reaches 76.7, against 70.0 for a weighted sum or sequential evolution and 53.3 for separately evolved harnesses. Per-task planning adds 10.7 points of secure pass for an 8% token overhead.
Abstract
Recent work on self-evolving LLM agents treats the agent harness as an optimization target that can be iteratively improved through execution feedback. Existing approaches, however, largely optimize a single objective and evolve the harness as a monolithic artifact, limiting their ability to handle competing goals and to separate transferable design from environment-specific adaptation. We introduce Harmony Harness, a framework for multi-objective, compositional harness evolution. Harmony Harness formulates harness evolution as a multi-objective optimization problem and uses decomposition-based evolutionary search to maintain solutions across different regions of the Pareto frontier, avoiding reliance on a hand-designed structural diversity metric for repository-scale harnesses. In parallel, we introduce factorized evolution operators that separately refine general-purpose harness mechanisms and benchmark-specific adaptations, together with a dynamic composition workflow that integrates them at inference time. Across SecRepoBench and BaxBench, Harmony Harness consistently outperforms strong harness-evolution baselines, with gains exceeding 10% on both benchmarks. Our results show that explicitly modeling both objective trade-offs and harness structure yields substantially stronger and more transferable agent evolution than existing single-objective, monolithic approaches.