Heres how to get rid of Nan exception errors for good

[2blackbar]

Hey Vlad heres some suggestion maybe youll be first to make it work cause auto11 is kinda busy with other stuff but this error is driving ppl crazy and this fix could be automated so its gone for good.
BEst result is when i switch to model2 and then back to model1 , this way i get lot of gens before nan exception will come back again.
Dont use xformers, it will block this solution, so use optimisation in settigns that will get you same speeds as xformers.
How i solved it is ( cause i had it A LOT, now i dont have it at all) i copied /cloned my model file so i have 2 copies of it just renamed v1 an v2.... then i just switch back and forth between them when i get Nan which is fast cause theyre in VRAM.

When i get Nan exception i switch to v2, when i get Nan again i swtich to v1, this way Nan eception is not an issue anymore ... ever.

SO... I dont want to be a smartass but obviously Nan exception never happens when you generate with a model after you just loaded it...
How about adding a code that will do same thing automatically and reflush current model before generating, so i dont have to keep 2 copies of the same model and keep oading them manually back and forth just to move on without Nan exception ?
A code that would automate it and would not require copy of current model.

Its not solving the core problem but makes the issue bearable, so whatever webui does when loading model from VRam ( i have it set so i keep 1 model in vram) would be nice to apply to the model once Nan exception is detected.

Its very annoying issue but this simple swtich of models from VRAM is making it go away.
Whad do You think? Is it doable ?
Id do it myself if id find a part of the code thats responsible for reloading model from VRAM when you choose it from the list but im not that smart with coding, maybe with the help of bing chat ... but probaly only as a switch to reflush, but id like it to be automatic when Nan exception is detected .
simple pull request with extra code that does model reflush like during swapping should solve this crappy error for good .

[vladmandic]

it only helps in a small number of nan scenarios - when they are interim.
in most cases, nan either doesn't happen or it happens always.
and my hands are pretty tied with trying to solve borderline cases as i don't have nan issues - anything i could reproduce, i've solved - what remains is stuff i cannot reproduce, so i cannot really move forward with solving it.

question: does unloading weights and then reloading them also help?

[2blackbar]

no sure how do you unload weights when youre running webui, id try that ,it works when you load another weight and load your weight back after that .
I get that its hard to fix things that dont happen to you, just wanted to leave it here so others will see how to deal with this crap, i got Nan almost all the time recently after like 50 generations and i was forced to close webui and restart it but this simple model switch and back solved this completely

[vladmandic]

in settings tab, top toolbar has buttons to unload and reload - can you try if that solves a problem for you. if yes, i can make that a bit more automated.

[2blackbar]

It doesnt help, i dont get nan exception in your webui but i get black image which is i guess supressed nan exception with no info in console, i was able to fix this by swapping to another model , no reload would help so its different than just realoading.

[vladmandic]

Thanks, I wanted to check that.

[2blackbar]

also xformers block this fix, i had to disable them , with them on its not working