Agentic ML Exploration (A-MLE) for Ads Ranking
A 38-author team at Meta Platforms reports Agentic ML Exploration, an autonomous LLM-agent system that runs the ML iteration cycle across a portfolio of production ads ranking models, and includes a controlled cross-LLM study of Claude Sonnet, Gemini and GPT families under a fixed agent loop.
Ask this paper
The bottleneck is human iteration throughput: Meta frames modern ads ranking as limited not by model capacity or training compute but by the number of research-implement-train-debug-evaluate-launch cycles senior engineers can run, each of which takes days to weeks per model.
Five stages under one orchestrating agent: A-MLE splits ML iteration into hypothesis generation, exploration strategy, experiment execution, result analysis and a shared knowledge substrate, orchestrated by a single agent that calls domain-specific skills and workflows against a sandboxed execution layer with human checkpoints at each stage boundary.
A tiered capability framework: Deployment is evaluated along three tiers: tool availability, autonomous workflow execution, and open-ended exploration, which separates an agent that can call the tools from one that can direct its own research.
Cross-LLM differences are qualitative: Holding the agent loop fixed, the model families differ in execution reliability and exploration aggressiveness, which is a distinction that aggregate benchmark scores do not surface.
The long tail is where it pays: The reported value is largest for the many ranking models that rarely receive expert attention, where proven techniques diffuse slowly.
Abstract
Modern industrial ads ranking stacks are increasingly bottlenecked not by model capacity or training compute, but by the throughput of human ML iteration - the cycles of research, implementation, training, debugging, evaluation, and launch required to surface a single statistically significant improvement. A typical ranking stack contains numerous differentiated models with heterogeneous data, architectures, and infrastructure constraints, and each cycle takes days to weeks of senior engineer attention per model. As a result, techniques that have proven effective on one model diffuse into others slowly and unevenly, leaving substantial recoverable signal unexplored. We present Agentic ML Exploration (A-MLE), an autonomous LLM-agent system that systematically explores ML techniques across a portfolio of ads ranking models. A-MLE decomposes ML iteration into five stages involving hypothesis generation, exploration strategy, experiment execution, result analysis and shared knowledge substrate which are orchestrated by a single agent that invokes domain-specific skills and agentic workflows against a sandboxed execution layer, with human-in-the-loop checkpoints at each stage boundary. We deploy A-MLE across a representative set of large-scale ads ranking models and evaluate it along a tiered capability framework (tool availability, autonomous workflow execution, and open-ended exploration). We further report a controlled cross-LLM study using a fixed agent loop, which surfaces qualitative differences in execution reliability and exploration aggressiveness across the Claude Sonnet, Gemini, and GPT families. We discuss failure modes and the design choices that govern reliability. Our findings suggest that agentic exploration is a practical force multiplier for ML engineers in industrial recommenders, especially for the long tail of models that rarely receive expert attention.