Question about the code in CrossAttention_Release.ipynb

[sunwoo76]

Hello,
Thank you for sharing your awesome code! :)

I have a question about this line:
latent_model_input = (latent_model_input / ((sigma**2 + 1) ** 0.5)).to(unet.dtype)

Could you give me some explanations about the reason that "(simga**2+1)**0.5" is needed ?

Thanks.

[bloc97]

That is for normalizing the latents before passing it to the u-net, as required for k-lms. This line is not required for all the other schedulers.