SimSkill: A Lifelong Learning AI Agent for Autonomous Mastery of Traffic Simulation
Qi Liu, Qinzheng Wang and Yiming Bie build SimSkill, a self-evolving agent over the SUMO traffic simulator that finds its own capability gaps, writes and solves grounded tasks, and consolidates the results into episodic, procedural and semantic memory without touching the backbone weights.
Ask this paper
Gap-driven curriculum: the agent identifies what it cannot do, generates environment-grounded tasks against that gap, and verifies solutions through an action-critic loop before consolidating.
Three memory types, measured separately: ablations show procedural and semantic memory contribute complementary gains, which is more informative than the usual single memory-on/memory-off comparison.
Up to 25 points on verified completion: evaluated on two held-out benchmarks with three backbone LLMs and independent artifact-based verification rather than self-report.
Honest about limits: the authors state that benefits are backbone- and budget-dependent, that memory does not help every model, and that it does not uniformly reduce inference cost.
Why it matters: a clean instance of the skills-library pattern where natural language stores the capability and executable code supplies reproducible execution.
Abstract
As large language models (LLMs) become increasingly capable, the long-term value of AI systems depends not only on solving individual requests, but also on transforming experience and accumulated knowledge into durable, reusable competence. We introduce SimSkill, a self-evolving agent built around the Simulation of Urban MObility (SUMO) traffic simulator. SimSkill identifies capability gaps, generates and solves environment-grounded tasks, verifies solutions through an action--critic loop, and consolidates experience into episodic, procedural, and semantic memory without updating the backbone model. Through autonomous exploration, it builds a reusable library spanning the traffic-simulation workflow. We evaluate SimSkill on two held-out benchmarks with three backbone LLMs and independent artifact-based verification. SimSkill improves verified completion by up to 25 percentage points, while ablations show complementary contributions from procedural and semantic memory. Its benefits remain backbone- and budget-dependent: memory does not improve every model or uniformly reduce inference cost. More broadly, SimSkill illustrates a design paradigm in which natural language preserves and composes computational capabilities, while executable tools and code provide precise and reproducible execution. All code and experimental data are publicly available at https://github.com/qiliuchn/SimSkill-V1.