Humanize MLE Agents: Multi-Agent Session Alternation Beyond Single-Agent Limits
Changye Li, Ligeng Zhu and colleagues (Tsinghua University, NVIDIA) present Humanize MLE Agents (HMA), which alternates two different agents on a shared workspace with capped sessions and context resets, and reaches 78.2% any-medal rate on MLE-bench.
Ask this paper
Long-horizon deceleration. As a single session runs longer, mean response length falls by 29.8% to 45.0% from hours 0-2 to hours 3-5, and progress slows with it.
Harness staleness. The benefit of custom MLE harnesses that manage tools, skills and experience shrinks as the underlying model improves, so harness gains may not carry over to the next model version.
Theory. Alternating sessions are modeled as a context-reset semi-Markov decision process, with sufficient conditions for alternation to beat single-agent limits and a bound on the experiment budget. A cap of k=5 experiments per session retains 97.83% of observed gains.
Results. On 75 Kaggle tasks, Opus-first HMA reaches 78.2% any-medal rate against 72.4% for the strongest single-agent baseline and 70.2% for the best published workflow, using a quarter of its time budget. HMA keeps 86.4% of its output tokens per response late in a run, against 60.7% for single agents.
Generality. Across six configurations of five agent pairs, HMA improves on the stronger constituent agent every time, by 6.4 points on average. A kernel-optimization case reaches 878 cycles, within 1.6% of the leaderboard leader.
Abstract
Existing LLM-based agent approaches for long-horizon machine learning engineering (MLE) often use context engineering, implemented through custom harnesses that manage tools, skills, and accumulated experience. However, we find that as single-session execution lengthens, the agent's reasoning intensity declines alongside slowing progress, summarized as long-horizon deceleration, and the benefits of these custom harnesses vanish as the underlying model improves, summarized as harness staleness. These findings suggest that their benefits may not persist across versions, which motivates a workflow-level approach in which agents build on one another's work across sessions. We model these alternating agent sessions as a context-reset semi-Markov decision process (CR-SMDP) and establish sufficient conditions for alternation to exceed single-agent reference limits, derive an explicit experiment-budget bound, and characterize when session caps preserve this advantage. Guided by this analysis, we introduce Humanize MLE Agents (HMA), which alternates two agents on a shared workspace, limits each session to at most k experiments and resets the context between sessions, without additional task-specific harness or prior knowledge supplements. Across 75 Kaggle tasks in MLE-bench, HMA reaches a state-of-the-art any-medal rate of 78.2%, exceeding the strongest open-source baseline by 8.0 points with a quarter of its time budget. Moreover, among six configurations spanning five heterogeneous agent pairs, HMA improves any-medal rate over the constituent baselines by 6.4 points on average, demonstrating cross-pair generality. A supplementary case on kernel optimization reaches 878 cycles, within 1.6% of the community leaderboard leader, demonstrating applicability beyond MLE.