Self Improvement via Fast Tree-search
Xinghong Fu, Aravinth Kulanthaivelu and Yutaro Yamada introduce SIFT, which identifies candidate evaluation as the runtime bottleneck in recursive self-improvement and replaces most of it with a judged tournament.
Ask this paper
Pairwise judging drives exploration. An LLM judge compares candidate patches, the win-loss record is aggregated with a regularized Bradley-Terry model, and the resulting strength scores drive rank-based parent sampling inside a disaggregated tree search.
Expensive evaluations are reserved for promising nodes. Only the most promising candidates get real downstream task runs, so the search is not gated by slow evaluation.
It beats prior tree-search self-evolution on Polyglot. With substantially lower CPU hours, wall-clock time and API cost, which is the constraint the paper sets for itself.
The bottleneck claim is the reusable part. Prior self-improvement loops estimate a modification's value by re-running benchmark subsets; naming that as the cost centre points at where any such loop should be optimised.
Abstract
Coding agents can recursively modify their own implementations, forming a loop of self-improvement. While prior work shows this can boost performance on coding benchmarks, existing approaches are costly and compute-intensive. We introduce a simple, sample-efficient self-improvement framework that significantly improves coding performance under strict budget constraints. We identify evaluation of candidate self-modifications as the main runtime bottleneck since prior approaches estimate their effectiveness by re-running a subset of benchmark tasks with the modified agent, which is time-consuming. We introduce Recursive Self Improvement via Fast Tree-search (SIFT), which augments these downstream task evaluations with an LLM-as-a-judge signal that performs pairwise comparisons between candidate patches, where the win-loss record is aggregated with a regularized Bradley-Terry model, and the resulting strength scores drive rank-based parent sampling inside a lightweight disaggregated tree search. Expensive downstream task evaluations are reserved only for the most promising nodes. Using a fully disaggregated tree search pipeline, the judge scores provide intermediate signal to guide exploration on promising candidate patches without being bottlenecked by slow evaluation runs. SIFT outperforms existing tree-search based self-evolution frameworks on the full Polyglot benchmark with significantly lower resource requirements in terms of CPU hours, wall clock time, and API cost.