Thanks for this great work!
But I find that there seems no weights synchronizing for diffusion models. As shown in figure, there is prepare_rollout_engine for models except for diffusion models, so I'm afraid that rollout engine of diffusion model always uses old weights to generate samples. I don't know whether I understand this correctly, looking forward for your reply, thank you a lot!
Thanks for this great work!
But I find that there seems no weights synchronizing for diffusion models. As shown in figure, there is prepare_rollout_engine for models except for diffusion models, so I'm afraid that rollout engine of diffusion model always uses old weights to generate samples. I don't know whether I understand this correctly, looking forward for your reply, thank you a lot!