Green image during inference.

[constan1]

I'm trying to train this on my custom dataset on faces. The core architecture of the model is roughly the same but I added a cross-attention block for conditioning on a different modality in addition to scale/shift for t. That being said. I see my MSE loss go down, but VLB is really unstable. However it does seem to converge.

Anyway, I'm training only a small subset to test. But I get really low mse loss and vlb after a while so the model seems to converge and learn the distribution to something but the output is just green.

If anyone could let me know what could be the reason for this? I'm using FP32.

in addition, I do normalize the images by dividing by 255 and then with normalize = transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])

its really odd that I'm getting specifically only green output