Question about Samplers? (Normal, simple, karras, DDIM Uniform)

[poisenbery]

Heyo, I've been a user of the Automatic1111 Webui for a while, I switched over to this when I realized how good it was.

I have a question about the sampler type: What is the difference between the "Normal, simple, karras, and DDIM Uniform" sample types?

I tried looking at the examples to see if I could spot a pattern in use cases; I noticed the "simple" sample type was used in the Img2Img type of examples, and Normal was used if it was the initial gen, but I'm not sure if this is the correct way for me to be interpreting these things.

[comfyanonymous]

Those are schedulers. They define the timesteps/sigmas for the points at which the samplers sample at. I decided to make them a separate option unlike other uis because it made more sense to me.

In karras the samplers spend more time sampling smaller timesteps/sigmas than the normal one. If you want the same behavior as other uis, karras and normal are the ones you should use for most samplers.

Simple is just one I added because I tried writing the simplest scheduler I could for fun and it actually worked well in some scenarios like second pass of hiresfix so I included it and named it simple.

ddim_uniform is the one that's supposed to be used with ddim if you want it to behave exactly like the ddim sampler in the reference Stable Diffusion implementation.

[poisenbery]

Thanks, I appreciate this.
Would it be possible to make a Wiki or some type of documentation that covers all of these things?
I used the VN tutorial but it seems like it's meant for beginners.

[NeedsMoar]

The stable diffusion source tree has links to all of the papers the samplers were implemented on. Diffusers usually uses a parameter on each sampler that effectively tells it which scheduler it should be using and a lot of other UIs will list the sampler / scheduler combinations as seperate entries on one list like "DPM++ 3M SDE Karras", "DPM++ 3M SDE Exponential", etc. I like the way comfy does it because it keeps the list from growing stupidly long with entries even though it took some getting used to because of the different naming.

[larinius]

ddim_uniform is the one that's supposed to be used with ddim

This is ONLY one combination which actually looks good and images are correct (no weird artifacts).
If user selects DDIM isn't it logical to display only working correct variants, instead of 20 confusing options which not work?
Poor user which has no idea after trial and errors will find correct combinations. Is it intentional behavior of UI? More like puzzle game.

[ltdrdata]

It’s not that simple. The combination is not only determined by the sampler and scheduler, but also varies depending on the model, and there are many other factors that can influence it.

There is even a technique where the scheduler, i.e. noise schedule, is intentionally altered (in a way, broken) to enhance the details.

[simonlui]

@comfyanonymous I don't want to start a new topic on this so I figured this would be the best place to ask. I see in comfy/k_diffusion/sampling.py that there are additional schedulers like polyexponential, vp, and etc. Additionally, some of the samplers themselves have additional parameters for what I assume are tuning knobs. Is there are particular reason or blockage on why some of the schedulers aren't implemented or some of the parameters aren't exposed to the user?

[NeedsMoar]

If I recall from the diffusers source some of the scheduler / parameter combinations are only meant for training for one thing.

[simonlui]

I dug a little more into this and it doesn't really make that much sense to me when other UIs have implemented polyexponential and vp as scheduler options. I may write something and test it myself. Will submit it as a patch for those options to be available if it works out.

[comfyanonymous]

If there's any sampler that benefits from those schedulers I'll add them.