SkillDuet: Co-Evolving Models and Skills through Response Feedback
Xingguang Ji and colleagues in Alibaba's Qwen Business Unit train a model and its library of textual skills together during RL, ranking skills by how much they change the current model's responses.
Ask this paper
Problem. During RL, skills extracted from experience can become redundant or harmful as the model's weights change.
Mechanism. Skills extracted from on-policy trajectories guide parameter updates through on-policy distillation at every step. A response-based score, Strength, ranks candidate skills by how much they shift the current model's responses, and the top skills are periodically consolidated into a persistent global set.
Results. With Qwen3-4B, SkillDuet reaches 98.57% on ALFWorld and 92.97% on WebShop, 2.86 and 5.47 points above the matched SEED-API baseline.
Ablations. Response-based selection and global consolidation each improve performance. Whether a retained skill helps depends on both its content and its compatibility with the current model.
Abstract
During reinforcement learning, experience can be both internalized in LLM parameters and retained externally as reusable textual skills. Parameter updates alter the model's behavior and the interaction experience from which new skills are extracted. These skills, in turn, guide the model's subsequent interactions and parameter updates. As the model's capabilities evolve, previously beneficial skills may become redundant or detrimental. We present SkillDuet, a framework that coordinates parameter updates and skill evolution through model response feedback. Skills extracted from on-policy trajectories guide parameter updates through OPD at every step. A response-based score, Strength, ranks candidate skills by the magnitude of the shift they induce in the current model's responses, and the highest-scoring skills are periodically consolidated into a persistent global skill set (Global). The updated model and Global jointly guide subsequent interactions, generating experience for further parameter and skill updates. With Qwen3-4B, SkillDuet achieves success rates of 98.57% on ALFWorld and 92.97% on WebShop, exceeding the matched SEED-API baseline by 2.86 and 5.47 percentage points, respectively. Ablations show that response-based skill selection and Global consolidation improve task performance. Further experiments show that the benefits of retained skills depend on both their content and their compatibility with the model. SkillDuet provides a concrete mechanism for model-skill co-evolution and skill-level recursive self-improvement.