GEPA

GEPA works by iteratively sampling trajectories from an LLM system, reflecting in natural language to identify issues, proposing new prompt edits, and combining successful strategies via a genetic Pareto search. It maintains a pool of diverse prompt candidates along a Pareto frontier to prevent local optima and encourage generalization.

Across four benchmarks, HotpotQA, IFBench, PUPA, and HoVer, GEPA outperforms the strong RL baseline GRPO by up to 20% and requires up to 35× fewer rollouts. It also surpasses the previous state-of-the-art prompt optimizer MIPROv2 by 10–14%, while producing shorter, more efficient prompts that generalize better across tasks and models.

A key innovation is GEPA’s use of reflective prompt mutation, where it explicitly uses an LLM to rewrite a module’s prompt based on failure traces and evaluation diagnostics. This enables targeted improvements after very few training examples, as visualized in optimization trees.

GEPA also introduces a system-aware merge strategy that combines independently evolved prompt modules from different lineages. While this improved performance with GPT-4.1-mini, gains were more modest with Qwen3-8B, highlighting the importance of model-specific tuning.

Finally, GEPA shows early promise as an inference-time search strategy. In code optimization benchmarks like NPUEval and KernelBench, it significantly boosts performance (e.g., from 4.25% to 30.52% vector utilization on NPUs) by reflecting on compiler errors and updating code-generation prompts accordingly.