Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation
Wenxuan Wang, Yang Yang and colleagues (Harbin Institute of Technology, Shanghai AI Laboratory, NTU, SJTU) propose D-OPCD, which distills the prompts improved by an agentic text-to-image harness back into the diffusion model weights.
Ask this paper
Harness gains. A skill-free agentic harness raises the four-benchmark average from 60.52 to 79.28, and skills evolved by their Auto Skill Evolver add 2.55 more, but these gains disappear when the harness is not running.
Distillation. D-OPCD treats the agent-improved prompt as privileged context and trains the diffusion model on-policy so that it produces similar images from the original query alone.
Result. Direct generation rises from 60.52 to 65.09 on average, 1.71 above vanilla SFT, leading on all four benchmarks, with +7.14 on WISE.
Co-evolution. With the knowledge absorbed, saturated skills can be dropped; a second skill-evolution round on the updated generator adds 1.83 points over a skill-free harness.
Direction. The authors propose harness and model improving each other in alternating rounds.
Abstract
Wrapping an image generation model in an agentic harness can effectively boost Text-to-Image task performance: the harness can leverage memory, skills, workflow orchestration, result verification, and iterative refinement to continually construct and revise prompts, thereby eliciting better images. These gains, however, remain external to the diffusion model and are realized only while the full harness runs. We propose Diffusion On-Policy Context Distillation (D-OPCD), which treats the agent-improved prompt as privileged context and distills the knowledge encoded in the agent harness into the weights of the diffusion model, so that the model retains part of the harness's benefit when conditioned on the original query alone. Using a Text-to-Image agent equipped with our proposed Auto Skill Evolver (ASE), we show that D-OPCD can internalize harness capabilities into the generator's weights, raising the average direct-generation score from 60.52 to 65.09 across four benchmarks. With this knowledge absorbed into the weights, the harness can shed its saturated skills and resume evolving: a second ASE round on the updated generator improves on a skill-free harness by additional 1.83 points, pointing toward text-to-image systems in which harness and model keep improving each other through continual co-evolution.