Does Comfy support batch prompts ran in parallel (like diffusers)? Single gpu forward pass

[billnye2]

Just wondering if comfy can run say 4 different prompts in the same gpu forward pass. Diffusers allows this by passing multiple prompts to the pipe, which are stacked in a tensor and pushed through the pipeline. The whole diffusers library operates under the assumption that the first dimension of the various tensors (prompt, negative prompt, seeds, etc.) is the batch, so you can run multiple at the same time with different settings.

Does comfy support anything like this?

[mcmonkey4eva]

yes just edit the batch_size in EmptyLatentImage

[billnye2]

That's only for running the same prompt through N times right? What about running 4 separate prompts all through in parallel

[billnye2]

E.g. running "a photo of a man" and "a photo of a woman" through one forward pass of the model

[comfyanonymous]

The core backend code supports it but it's not exposed in the interface. I think there are some custom nodes that support it. Basically you just need to modify the CLIP text encode node to return a multi batch tensor.