Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

Researchers have identified a significant knowledge gap in on-policy diffusion distillation, specifically regarding classifier-free guidance, a crucial component of modern diffusion systems. The current approach to on-policy distillation involves adapting diffusion models by querying a teacher along trajectories generated by the current student, but the behavior of this process under classifier-free guidance is not well understood. Existing methods extend velocity matching to the classifier-free guidance-composed prediction, directly matching teacher and student guided outcomes. However, this may not be sufficient, as the interaction between on-policy distillation and classifier-free guidance can have significant implications for the resulting model's behavior¹. The shift in threat models from criminal to geopolitical due to state-aligned activity involving diffusion models requires a different approach. This research aims to address the shortcomings of current on-policy distillation methods, which is crucial for practitioners working with diffusion models, as it can impact the security and reliability of their systems.