Holding a Steady Course: Knowledge Evolution from Expertise and Lessons for Continual Agent Learning
Wenzhe Fan, Benjamin Steenhoek, Gaurav Mittal and Yu Hu (Microsoft, University of Illinois Chicago) introduce KEEL, an agent-agnostic framework that uses verifier evidence to turn successful behavior into skills and failures into corrective lessons, and keeps revising both as evidence accumulates.
Ask this paper
Problem. Automatically collected skills and memories can carry errors forward and corrupt useful knowledge, because recorded experience is not automatically reliable.
Mechanism. Verifier evidence decides what becomes a skill or a lesson. Each skill claim can later be supported, contradicted, replaced or retired as new evidence arrives.
Results. On the SkillFlow benchmark, KEEL improves task success by 4.2 to 21.7 points over a no-knowledge baseline, against 2.4 to 9.6 points for SkillFlow's default method.
Ablations. Skills and lessons are complementary; removing either reduces the gain. Code is released at github.com/microsoft/KEEL-TTCL26.
Abstract
Autonomous agents are increasingly deployed across sequences of related tasks, yet they often fail to retain and improve their knowledge over time. Existing approaches convert interaction trajectories into memories or reusable skills, but experience is not automatically reliable knowledge. Automatically collected skills and memories can propagate errors, mislead, and corrupt useful knowledge. To address this gap, we introduce Knowledge Evolution from Expertise and Lessons (KEEL), an agent-agnostic framework that uses verifier evidence to transform successful behavior into reusable skills and failures into corrective lessons. As new evidence accumulates, individual skill claims can be supported, contradicted, replaced, or retired. On SkillFlow benchmark, KEEL improves task success by 4.2 to 21.7 percentage points over the no-knowledge baseline, compared with gains of 2.4 to 9.6 points from SkillFlow's default method. Ablations show that skills and lessons are complementary. These results show that reliable continual agent learning depends not simply on accumulating experience, but on continually validating and evolving the knowledge derived from it. Our code is available at: https://github.com/microsoft/KEEL-TTCL26.