ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Stabilize
Lihao Liu, Peng Tang, Kunwar Yashraj Singh and Shabnam Ghadar (AWS Agentic AI) trace GEPA-style prompt bloat to three specific deficiencies and fix each with a named phase, producing prompts 47% shorter that score higher.
Ask this paper
Bloat is a symptom, not the disease: evolutionary optimizers append rules each iteration, producing prompts up to 3x longer with no accuracy gain. The causes are incomplete error observation, limited search diversity and unreliable selection.
One phase per deficiency: Diagnose clusters all training errors into structural patterns in a single round, Propose generates candidates via four complementary strategies with independent biases, and Select applies bootstrap stability selection.
+3.76 points over state of the art: 74.67% vs 70.91% for GEPA across Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer and PUPA, matching or beating GEPA on every single dataset.
Shorter and faster: 1,004 characters against 1,878, so inference cost falls alongside accuracy rising.
The ablation confirms a predicted interaction: adding diversity without bootstrap selection actively hurts, by 1.20%, which is the kind of falsifiable prediction most prompt-optimization papers do not make.
Abstract
Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends rules and caveats, producing prompts up to 3$\times$ longer yet no more accurate. We trace this to three deficiencies - incomplete error observation, limited search diversity, and unreliable selection - and propose ESPO (Error-Structured Prompt Optimization), which decomposes prompt optimization into three phases: Diagnose clusters all training errors into structural patterns in one round; Propose generates candidates via four complementary strategies with independent biases; Select applies bootstrap stability selection. On seven public NLP benchmarks - Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, and PUPA - ESPO improves average accuracy by $+$3.76 pp over the state-of-the-art (74.67% vs 70.91% for GEPA), matching or exceeding GEPA on every dataset while producing prompts 47% shorter (1,004 vs 1,878 chars) and faster at inference. Cross-model experiments across four additional student models (Gemma 3 12B, Mistral 14B, Qwen3 32B, Claude Haiku 4.5) show ESPO yields the best average accuracy on every model tested, with the largest gap on Qwen3 GSM8K (15.00% $\to$ 91.40%). A generalization bound (Appendix) grounds each phase in a corresponding term of the test-time gap, and the ablation confirms a key prediction: adding diversity without bootstrap selection actually hurts performance ($-$1.20%).