GEPA
Introduces a new optimizer, GEPA, that adaptively improves prompts for compound AI systems using natural language reflection and Pareto-based search. Rather than relying on reward gradients from traditional RL, GEPA explicitly reasons over LLM execution traces and feedback to evolve better prompts, dramatically increasing sample efficiency and final performance.
Ask this paper
GEPA works by iteratively sampling trajectories from an LLM system, reflecting in natural language to identify issues, proposing new prompt edits, and combining successful strategies via a genetic Pareto search. It maintains a pool of diverse prompt candidates along a Pareto frontier to prevent local optima and encourage generalization.
Across four benchmarks, HotpotQA, IFBench, PUPA, and HoVer, GEPA outperforms the strong RL baseline GRPO by up to 20% and requires up to 35× fewer rollouts. It also surpasses the previous state-of-the-art prompt optimizer MIPROv2 by 10–14%, while producing shorter, more efficient prompts that generalize better across tasks and models.
A key innovation is GEPA’s use of reflective prompt mutation, where it explicitly uses an LLM to rewrite a module’s prompt based on failure traces and evaluation diagnostics. This enables targeted improvements after very few training examples, as visualized in optimization trees.
GEPA also introduces a system-aware merge strategy that combines independently evolved prompt modules from different lineages. While this improved performance with GPT-4.1-mini, gains were more modest with Qwen3-8B, highlighting the importance of model-specific tuning.
Finally, GEPA shows early promise as an inference-time search strategy. In code optimization benchmarks like NPUEval and KernelBench, it significantly boosts performance (e.g., from 4.25% to 30.52% vector utilization on NPUs) by reflecting on compiler errors and updating code-generation prompts accordingly.
Abstract
Large language models (LLMs) are increasingly adapted to downstream tasks via reinforcement learning (RL) methods like Group Relative Policy Optimization (GRPO), which often require thousands of rollouts to learn new tasks. We argue that the interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt optimizer that thoroughly incorporates natural language reflection to learn high-level rules from trial and error. Given any AI system containing one or more LLM prompts, GEPA samples trajectories (e.g., reasoning, tool calls, and tool outputs) and reflects on them in natural language to diagnose problems, propose and test prompt updates, and combine complementary lessons from the Pareto frontier of its own attempts. As a result of GEPA's design, it can often turn even just a few rollouts into a large quality gain. Across six tasks, GEPA outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts. GEPA also outperforms the leading prompt optimizer, MIPROv2, by over 10% (e.g., +12% accuracy on AIME-2025), and demonstrates promising results as an inference-time search strategy for code optimization. We release our code at https://github.com/gepa-ai/gepa .