Auto-RecSys: Harnessing Autonomous Research Agents for Industry-Scale Recommender System
Ming Li, Dai Li and colleagues at Meta build Auto-RecSys, an autonomous research agent system that runs multi-day experiments on industry-scale recommendation models, built around three harness designs and two self-improvement loops.
Ask this paper
Two obstacles: Training runs take days, so serial iteration is too slow and parallel exploration is needed. Large configurations, fragile infrastructure and multi-day GPU jobs require recoverable execution.
Harness designs: Distributed asynchronous execution runs experiments in parallel across servers. Centralized cross-server memory makes runs recoverable across sessions and failures. Natural-language skill files guide reasoning while deterministic scripts enforce operational correctness.
Execution Evolution Loop: Model-specific playbooks record failed attempts and keep successful pipelines, so operational knowledge accumulates.
Idea Evolution Loop: Experimental outcomes feed back into the next round of hypothesis generation.
Outcome: Human time per experiment cycle falls and execution reliability rises as the playbooks mature on production recommendation models.
Abstract
Auto-research agents have shown the potential to automate hypothesis generation, experiment execution, and iterative refinement. However, scaling this paradigm to industry-scale recommendation models introduces two challenges: (1) long feedback loops, where model training can take days, making serial iteration prohibitively slow and requiring parallel exploration across multiple research directions; and (2) system complexity, where large configurations, fragile infrastructure dependencies, and multi-day GPU jobs require robust and recoverable execution. We present Auto-RecSys, an autonomous research system for long-horizon experimentation on industry-scale recommendation models. Auto-RecSys addresses these challenges through three harness designs: (1) distributed asynchronous execution for running multiple experiments in parallel across servers, (2) centralized cross-server memory for persistent and recoverable execution across sessions and failures, and (3) cognitive-procedural separation, where natural-language skill files guide LLM reasoning while deterministic scripts enforce operational correctness. Auto-RecSys further employs a dual-loop self-evolving architecture: an Execution Evolution Loop in which model-specific playbooks accumulate operational knowledge by recording failed attempts and crystallizing successful pipelines, and an Idea Evolution Loop in which experimental outcomes inform subsequent ideation. Evaluated on recommendation models, Auto-RecSys significantly reduces the human time required per experiment cycle and improves execution reliability as its playbooks mature.