Is it possible to use opt-sdp-attention instead of xformers? #293
-
|
Hi, would it be possible to use opt-sdp-attention optimisation instead of xformers? Ive switched to torch2 and everything is running fine, however I do get better performance using opt-sdp-attention, would it be relatively easy to change comfyui to use it as well. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 6 replies
-
|
Yes, the command line option to use it is: --use-pytorch-cross-attention The reason I don't really advertise it is because according to tests performance is worse than xformers for most people at higher resolutions where speed is usually more important than at lower resolutions. |
Beta Was this translation helpful? Give feedback.
-
Have you seen how bad xformers destroys images though, and is totally not meant for any for production? It's definitely a gimmick just for low quality images you don't care about. Take a xformers image and level it, or curves it and you'll see how bad it's seams are, all requiring post processing for anything serious. IMO neither should be defaulted to on unless users enabled them, because both methods aren't appropriate for any serious work. Also one of the chief ways images are determined to be AI is their tiling seams if not using an AI model. |
Beta Was this translation helpful? Give feedback.
-
|
In ComfyUI it should not "destroy the image", at least that hasn't been my experience when I was comparing things. It should give outputs that are pretty much impossible to tell apart with human eyes even if you put the images on top of each other to compare them. If it does destroy the image it's a bug and should be reported. |
Beta Was this translation helpful? Give feedback.
-
It's not with the human eye though, it's stuff that contaminates post-processing work. However, after inspecting a rather large image (4k) I actually don't see any seams. I do have xformers active, I think, in comfyui (default settings for portable). That's rather impressive. This is what I deal with in webui when xformers is on: (Here is a me as an examplee) It destroys images. All those bucket seams need post processing to be able to apply different filter adjustments and composite them. Perhaps webui is defaulting to a tiled (and bad VAE), but it seems to only appear with xformers, and to a lesser degree with cross attention fix. |
Beta Was this translation helpful? Give feedback.
-
|
If you see "Using xformers cross attention" in the ComfyUI console that means xformers is being used. If you are talking about the a1111 webui the code quality is quite bad and it's most likely a problem with the UI code itself rather than xformers. |
Beta Was this translation helpful? Give feedback.
-
Sounds like it. I always assumed it was xformers or cross attention cause they both created the effect, though xforms seemed more right, which meant it was a little tougher to isolate and tone out the anomalies. Actually really liking the performance, and quality. Even toled VAE is really nice now. |
Beta Was this translation helpful? Give feedback.
-
|
do i add this argument when I run main.py? Like so? py main.py --use-pytorch-cross-attention If this is the case I did this and noticed zero difference in image and the iterations/s were identical. |
Beta Was this translation helpful? Give feedback.
-
|
i'm also curious if this is how i enable flash_attn... cos innitially i thought simply installing flash_attn i shaved off more than 60% of time waiting... but... when i tried to reinstall a fresh new install of comfyui... i am actually running many times faster than before suggesting that i did the original installation badly and messed up. but not sure where i did wrong... |
Beta Was this translation helpful? Give feedback.
Yes, the command line option to use it is: --use-pytorch-cross-attention
The reason I don't really advertise it is because according to tests performance is worse than xformers for most people at higher resolutions where speed is usually more important than at lower resolutions.