OPRO (LLMs as Optimizers)
DeepMind's OPRO uses LLMs as general-purpose optimizers over natural-language-described problems.
Ask this paper
Natural-language optimization: The optimization problem is described in natural language; the LLM iteratively proposes new solutions conditioned on previously found solutions.
Prompt optimization: As a key application, optimizes prompts to maximize test accuracy, using previously evaluated prompts as trajectory context.
Big gains over human prompts: LLM-optimized prompts outperform human-designed prompts on GSM8K and BIG-Bench Hard, sometimes by over 50 percentage points.
General-purpose pattern: Positions LLMs as general-purpose optimizers for problems that are hard to specify mathematically, including linear regression, traveling salesman variants, and prompt design.