Evolving Agent Teams
Shiyi Cao, Ziming Mao, Dacheng Li, Joseph Gonzalez and Ion Stoica (UC Berkeley) introduce Evolving Agent Teams, a multi-agent framework in which a supervisor restructures a team of coding agents whenever optimization stalls, and apply it to production GPU kernels.
Ask this paper
Problem. Existing coding agents converge to local optima on expert-written kernels such as FlashInfer.
Design. A central supervisor diagnoses each local optimum and restructures the team. A persistent file system holds a learning document, an evolution history and per-team artifacts, so restructuring decisions use accumulated experience.
Kernel results. Under a matched token budget it beats strong coding-agent baselines. With a larger budget it writes from scratch a CUDA kernel with 1.2x mean speedup over FlashInfer's production MLA paged decode kernel, winning on 36 of 47 FlashInfer-Bench workloads (0.86 to 0.99x on the rest).
Other tasks. It reaches 3.01x over vLLM on KDA prefill on AMD MI350X, 3.44x over FlashKDA on NVIDIA B200, 30.3 tokens/s MoE expert-offload decoding on a 48 GB M4 Pro, and 62.1 on held-out RoboDojo seeds against 55.3 for the best leaderboard entry.
Abstract
The rapid progress of software engineering agents drives interest in applying them to open-ended problems such as GPU kernel optimization. However, we find that existing coding agents converge to local optima on real-world, expert-written kernels (e.g., FlashInfer). We introduce Evolving Agent Teams, a multi-agent framework that escapes such local optima. It features (1) a central supervisor that diagnoses each local optimum and restructures a team of agents; and (2) a persistent file system that comprises a learning document, an evolution history, and per-team artifacts, grounding restructuring with accumulated experience. Under a matched token budget, Evolving Agent Teams consistently outperforms strong coding-agent baselines. With an extended token budget, Evolving Agent Teams generates a CUDA kernel that reaches 1.2x mean speedup over the production FlashInfer MLA paged decode kernel from scratch and wins on 36 of 47 production workloads from FlashInfer-Bench (0.86-0.99x on the remaining 11). It also demonstrates superior performance on tasks such as KDA prefill on AMD MI350X (3.01x over vLLM) and NVIDIA B200 (3.44x over FlashKDA), MoE expert-offload serving on Apple silicon (30.3 tokens/s decode for Qwen3.8-Flash-Next in MLX 4-bit on a 48 GB M4 Pro), and RoboDojo robot manipulation (62.1 on held-out seeds vs. 55.3 for the best leaderboard entry). Code is available at https://github.com/caoshiyi/EAT.