First frame in animation is too noisy after update

[rsndetre]

after update (25.01.2025) Comfy and AnimateDiff-Evolved first frame in animation is too noisy

after returning to versions from 05.01.2025 animations become normal as they were before (AnimateDiff: git checkout 63b70f1 ; ComfyUI: git checkout caa6476a699f04ca1cd269a03423510b1a360d22)

example
before update (25.01.2025)

after update (25.01.2025)

Thank you for all your hard work!

[Kosinkadink]

Do these images have workflows embedded? If not, could you make a simple workflow that replicates the issue you are seeing?

[rsndetre]

Do these images have workflows embedded? If not, could you make a simple workflow that replicates the issue you are seeing?

Thank you for your quick reply! I will reply to you a bit later. I will experiment with simple workflows. My workflows are too complicated and overloaded with other nodes.

[rsndetre]

simple workflow that replicates the issue
after update

before update

after update

before update

[rsndetre]

pay attention to Sample Settings 🎭🅐🅓/seed_gen/gpu
maybe the problem is here I will do some more tests

[rsndetre]

on Sample Settings 🎭🅐🅓/seed_gen/comfy[gpu] all the frames are just noise
after update

before update

on Sample Settings 🎭🅐🅓/seed_gen/comfy there is no noise visible but all the frames change to others
after update

before update

[Kosinkadink]

After trying out the workflows for a bit, I've narrowed down the issue - it only affects ancestral samplers. For example, if you switch to euler (or any other non-ancestral sampler), you will not see the noise issue whether you use gpu noise or not. (and before and after results are likely the same, but i haven't tried).

There was a change in ComfyUI somewhat recently to allow ancestral samplers to have more reproducibility across a batch, but it looks like that has had a side effect. Somehow it breaks how I implemented GPU noise, I'll investigate some more and get back to you.

[Kosinkadink]

comfyanonymous/ComfyUI@916d1e1 Looks like someone noticed the issue with ancestral samplers when using GPU noise in another node pack as well after the commit that did this.

[Kosinkadink]

Even rolling back to the commits you wrote in your initial post, i can't get your 'before' results. Can you double check what commits you are on exactly for your 'before' results?

[Kosinkadink]

Also, please post the full animations as well - if you make the uploads h264, they should be postable.

[Kosinkadink]

To simplify testing, you can also leave the batch size at 8, so you dont trigger any contexts - the noisy results are reproducible with that too.

[Kosinkadink]

interestingly, you can also replicate the weird noise if you even turn off AD and just do a single image of SD

[Kosinkadink]

I did a fresh install of ComfyUI with just AnimateDiff-Evolved and VHS installed, and set them both back to October 24th commits, and i still get back the noisy results. At this point I am at a loss. Can you provide as much info as you can about your environment when you are in the 'before' phase? Your OS, pytorch versions, etc.

[Kosinkadink]

FYI, there is no reason to use the GPU noise in terms of quality, so for the time being you can just change to non-GPU noise and you should be fine. But this is very odd that I am unable to reproduce your 'before' results, or that GPU noise is acting very weird when combined with ancestral samplers. I will try to go even further back in commits.

[Kosinkadink]

Review time: while I am not able to reproduce your 'before' results, I have figured out the exact cause for the bad results when whatever [gpu] seed_gen is in use.

'Usual' samplers do not apply any new noise during sampling (there is only the initial noise), while ancestral samplers apply random noise every single step. That noise has to be seeded somehow. The way this is done is inside the default_noise_sampler function - since the x here is on the GPU rather than CPU, the generator will use the GPU to create noise:

This means that when seed_gen is whatever [gpu], the first ancestral noise will be identical to that of the initial noise, which completely screws up how that is expected to work. Specifically, the ancestral noise is equivalent to comfy [gpu], which makes the first frame's noise the same as auto1111 [gpu], which is why if you run a single image right now, you will get identical noisy results, while if you do an AD run, auto1111 [gpu] will be slightly better but still noisy as hell.

To confirm this for yourself, run one of the bad, noisy workflows that uses whatever [gpu] seed_gen but simply change seed_offset to something that is not 0. This will make the initial noise used seed+seed_offset instead of seed, while the ancestral sampler code will still use seed, giving you expected clean results:

That default_noise_sampler can be cleanly overriden, so what I am going to do is add an ancestral_opts input into Sample Settings that will take in an Ancestral Options node, where you can explicitly set ancestral seed stuff like a simple offset or seed override. I will update this issue when I have that implemented.

[Kosinkadink]

I just pushed an update to add Ancestral Options:

You can now offset (or specifically choose) the seed used for the ancestral noise, so that you can avoid running into that clash of ancestral noise and initial noise for gpu-type seed_gens.

[rsndetre]

I did a fresh install of ComfyUI with just AnimateDiff-Evolved and VHS installed, and set them both back to October 24th commits, and i still get back the noisy results. At this point I am at a loss. Can you provide as much info as you can about your environment when you are in the 'before' phase? Your OS, pytorch versions, etc.

Win10 pytorch version: 2.5.1+cu121
Python version: 3.11.9

To confirm this for yourself, run one of the bad, noisy workflows that uses whatever [gpu] seed_gen but simply change seed_offset to something that is not 0. This will make the initial noise used seed+seed_offset instead of seed, while the ancestral sampler code will still use seed, giving you expected clean results:

at first glance it did not bring the desired result, the noise became of a different character but it is there. (just changed seed_offset to -1, but left Auto1111(GPU) ). I will continue experiments (I haven't used today's update yet)

non-ancestral euler works as expected. Identical before and after update

FYI, there is no reason to use the GPU noise in terms of quality, so for the time being you can just change to non-GPU noise and you should be fine. But this is very odd that I am unable to reproduce your 'before' results, or that GPU noise is acting very weird when combined with ancestral samplers. I will try to go even further back in commits.

Yes, I know about it (in theory). Just when I switched to comfy I reproduced the results from A1111 and since then I left this setting

comfyanonymous/ComfyUI@916d1e1 Looks like someone noticed the issue with ancestral samplers when using GPU noise in another node pack as well after the commit that did this.

I pasted the code before this commit into sampling.py after update and it gives clean results (but a little different)

[rsndetre]

Even rolling back to the commits you wrote in your initial post, i can't get your 'before' results. Can you double check what commits you are on exactly for your 'before' results?

Yes, everything is as in initial post
before is:
Comfy: git checkout caa6476a699f04ca1cd269a03423510b1a360d22
AnimateDiff: git checkout 63b70f1

Also, please post the full animations as well - if you make the uploads h264, they should be postable.

h264
before
https://github.com/user-attachments/assets/2ab47fc3-ea5d-4385-a91e-6051808e62e9
after
https://github.com/user-attachments/assets/71a43101-4f5a-4fe5-bf67-941dd48b7b60
before
https://github.com/user-attachments/assets/a2cd0a29-d6f4-4e0b-8f34-ecf133c8ee5e
after
https://github.com/user-attachments/assets/bcecfadf-930f-4074-8c45-5ba8e9e6497b

[rsndetre]

Даже откатываясь к коммитам, которые вы написали в своем первоначальном посте, я не могу получить ваши результаты "до". Можете ли вы перепроверить, какие именно коммиты вы выполняете, для ваших результатов "до"?

I updated to pytorch version: 2.6.0+cu124 and its behavior is similar to 2.4.1+cu121 - before the update - clean, after the update - noise.
Today's patch works well, clean and smooth. Thank you for quickly resolving the problem!
But of course, the problem in Comfy remains (without AnimateDiff), now the image generation (without AnimateDiff, with noise_mode - GPU(A1111) and sampler - euler_ancestral) is broken - in batch 1 just noise is generated, and in batch more than 1 - the first image is noise.