Dual GPU

[Robson1970]

How to run on multiple GPUs ?

[Ferniclestix]

You can tell comfyui to run on a specific gpu by adding this to your launch bat file.
set CUDA_VISIBLE_DEVICES=1 (change the number to choose or delete and it will pick on its own)

then you can run a second instance of comfy ui on another GPU.

there is a network custom node thing that lets you send instructions to other comfyui instances on the local machine or network but I dont really know much about it.

Suffice to say its generally not really supported in the ui.
I'd love if it was, especially for SDXL, then i could run each model on a different gpu, it would be sooo goood and you might not have to dump to system ram so much.

[maxchiron]

You can tell comfyui to run on a specific gpu by adding this to your launch bat file. set CUDA_VISIBLE_DEVICES=1 (change the number to choose or delete and it will pick on its own)

then you can run a second instance of comfy ui on another GPU.

there is a network custom node thing that lets you send instructions to other comfyui instances on the local machine or network but I dont really know much about it.

Suffice to say its generally not really supported in the ui. I'd love if it was, especially for SDXL, then i could run each model on a different gpu, it would be sooo goood and you might not have to dump to system ram so much.

Won't work now cause main.py would check the init args and reset the CUDA_VISIBLE_DEVICES.
So the working way now is:

python main.py --cuda-device 3

or add the --cuda-device arg to the .bat file's python's line.

[Ferniclestix]

recent update? haven't updated for two days.

[Ferniclestix]

huh, nope, seems to still work, just updated.

[nothingness6]

You can tell comfyui to run on a specific gpu by adding this to your launch bat file. set CUDA_VISIBLE_DEVICES=1 (change the number to choose or delete and it will pick on its own)

then you can run a second instance of comfy ui on another GPU.

there is a network custom node thing that lets you send instructions to other comfyui instances on the local machine or network but I dont really know much about it.

Suffice to say its generally not really supported in the ui. I'd love if it was, especially for SDXL, then i could run each model on a different gpu, it would be sooo goood and you might not have to dump to system ram so much.

Hi, could you tell me where the launch.bat file is? I'm using Stability Matrix and can't find it at all.

[nothingness6]

Thanks for the response. I added it and faced the error, RuntimeError: No CUDA GPUs are available

[brknsoul]

What GPU do you have?

[nothingness6]

RTX3060 6GB on laptop.

[brknsoul]

try setting it to 0, instead of 1. Some laptops can be weird.

[nothingness6]

Ok, the error message has gone, but it seems nothing changed. Probably, my laptop is one of the weird ones :(
I'm running the following workflow and it gives me Out of Memory. Could you tell me how to solve this issue, please?

animatediff-v2v-controlnet-openpose-HED-LCM-LoRA-detailer2.json

[h3llolovely]

Edit
ComfyUI/comfy/cli_args.py

Search for "--cuda-device"
Replace "default=None" to "default=1" (or whichever gpu you want to select. i.e 0-4+)

Good to go.

[brknsoul]

rather than editing, couldn't you just add --cuda-device 1 to Extra Launch Arguments at the bottom of Launch Options?

[h3llolovely]

If it works, sure.
I'm on a local install using pinokio.io
The start.json file didn't take that arg for me.

[brknsoul]

for local installs, you typically start ComfyUI with python main.py --args --go --here

[h3llolovely]

I tried that and it didn't work for my install.
If you have a run.bat (I think it's called), you can pass the args there as well.

[unphased]

https://github.com/Stability-AI/StableSwarmUI

[rbychn]

can this run on multiple GPUs at once? can we load any comfyUI workflow onto this?

[unphased]

yes

[rbychn]

just got to know it can only run 1 inference on 1 GPU at any given time, i wanted to speed it up by running 1 job on multiple GPUs but it can't do it unfortunately

[unphased]

Yeah diffusion models can't run efficiently spread across more than one gpu for the generation of a single image. But generating more than one image is a common use case. I would like to imagine that there could also be gains available from multiple gpus when rendering tiles (e.g. ultimate SD upscale). Same for video generation (SVD, AnimateDiff).

[drmbt]

when launching with --cuda-device 1, i see:

Set cuda device to: 1
Total VRAM 24564 MB, total RAM 65186 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync

it still won't run on second card. Anything I'm missing?

[Ch-Shi]

I am in the same situation as you, I checked the gpu usage and found that I was indeed using a second graphics card, so I think it is a display error of the controlled print

[unphased]

you guys are confusing me. to generally use the two gpu's easily, you should run two instances, one specifying device 0 and one specifying device 1. You seem to be talking about having been able to specify and run on device 1 so go ahead and do so on device 0 as well.

[drmbt]

When I run two instances, with the second argued for my second GPU, that seems like it's working there, although the printout is still wrong.

My problem is that I'm trying to run a single instance on the second GPU in an art installation setting, with my first default GPU powering unreal engine and TouchDesigner, offloading my SD comfy stack to the second GPU. So far, it doesn't seem to want to do this unless I have the first GPU already occupied with a comfy instance, which is what I'm trying to avoid

[nono-london]

Hi, not sure I am picking up the right thread...
I am a newbie in this, so I may be completely wrong.
I want to run ComfyUI with 3 graphic cards. First one is on my PCIx16, the 2 others are on risers. Idea is to compensate the lack of memory needed for FLUX1 - especially the controlnet from XLab (FLUX1 works as it is, even in the dev version).

As for now, when I run ComnfyUI, it only recognise my PCI16 card, but not my 2 other risers (which are recognized at the Windows level.
I use ComfyUI on WSL2, with python 3.10.
If this can be done, is there a step by step tutorial?
Thanks a lot for the help.

Best

[ltdrdata]

That's not possible. Currently, the only way to use multiple GPUs is to run multiple instances of ComfyUI.
And this method cannot be used to increase the total amount of VRAM.

[nono-london]

Thanks a lot for clarification, I was wondering if maybe this project handling another type of models could partly do that: ComfyUI-GGUF. Thanks again, sorry if the questions seem futiles. Best

[ltdrdata]

Thanks a lot for clarification, I was wondering if maybe this project handling another type of models could partly do that: ComfyUI-GGUF. Thanks again, sorry if the questions seem futiles. Best

While I hope for development in that direction, it won't be easy to achieve in the short term.

[ghost]

I think this has been answered.
#4139