← All papers  /  Nov 14, 2026
Agents · Evaluation · Efficiency

CompoundingKernel: Self-Improving from Zero to First

First page
CompoundingKernel: Self-Improving from Zero to First
The curator’s take

A team at Amazon presents CompoundingKernel, a self-improving search system in which over a thousand coding agents write GPU kernels from scratch, and whose kernels rank first on the public L1, L2 and Quantization boards of SOL-ExecBench on NVIDIA B200.

Ask this paper

Key points
01

Results. All 235 problems have a site-verified correct kernel. On the 209 public-board problems the kernels are 4.40x faster than the benchmark's reference baseline, and they have held first place on L1 since 2026-10-27 apart from a short loss on 2026-11-07.

02

From zero. Agents get each problem's PyTorch definition but no reference kernels or official scores. Official scores never enter the search loops, which learn only from the team's own tests and timings.

03

Four loops. A child agent edits one kernel in one direction and checks outside the model decide correctness. An LLM controller reads a per-problem archive of every correct kernel and picks the next parent. A team of agents fixes the harness at warm restarts. People decide what to submit.

04

What limits scale. At equal wall time the LLM controller beat a UCB-style tree search on 35 of 38 problems by making 2.4x as many correct attempts. One warm-restart fix raised the fleet's evaluation rate 3.6x. The rate of reliable measurements limits the search, not the number of agents.

Abstract

We present CompoundingKernel, a self-improving agent system that writes GPU kernels by search. As of 2026-11-11, its generated kernels rank #1 on the public L1, L2 and Quantization boards of SOL-ExecBench (NVIDIA B200). They have held #1 on L1 since 2026-10-27, albeit a short loss on 2026-11-07. All 235 problems have a site-verified correct kernel; on the 209 public-board problems they are 4.40x faster than the benchmark's reference baseline. The search starts from zero: agents get each problem's PyTorch definition but no reference kernels or official scores. A per-problem archive outside the model holds every correct kernel and its lineage. An LLM controller reads it, picks a parent or a new design, and gives each parallel child agent one direction. Checks outside the model decide correctness. At equal wall time, it beat a coded UCB style tree on 35 of 38 problems by making 2.4x as many correct attempts. Over a thousand agents ran at once, and one warm restart fix raised their evaluation rate 3.6x. We find that the rate of reliable measurements, not the number of agents, limits how far such a search scales. Our code and kernels are available on URL.

Every Monday
Get next week’s papers.
Subscribe on Substack