A thank you... and may be an idea

[AceroMagico]

I read on reddit a thread about your project and saw how many people asked questions and made suggestions about ComfyUI.
This is certainly helpful to you and the project, and me too in a few lines below I will ;D
But maybe first, it would be fair to Thank You for the work you have done and for making it available to everyone.

I think your approach and your choice of using nodes as the interface system is the right way and certainly the one that everyone will adopt in the future because it is flexible, easily modifiable and fast but very compact compared to other auto1111-type like interfaces.

I really appreciated the possibility of not having to install dependencies if you already have another python interface, and I find the idea of embedding the project data inside the drag and drop final images, truly ingenious!

So again thank you very much.

Having said that, I will pass on a suggestion that if you are willing to follow it could bring about may be, a new possibility of using AI generated images in a professional way, but in theory it is not very complicated to achieve.

Chroma key, or green screen, is a technique widely used in video and image editing to isolate a subject from its background by replacing a specific color (usually green or blue) with a different image or video.
In the context of neural network-based image generation, this technique could be leveraged to separate the subject from the background with precision and reproducibility, simplifying the compositing process overcoming the problem that the images in the neural network process must be a two-dimensional matrix of continuous pixels.

No imaging professional can do anything without alpha mask channels.
And one of the main problems at present in AI images generation is the reproducibility of the parts of the image that are judged to be creative valid.

Of course we can put a fixed seed and always get the same output image with the same initial parameters, but if you have the generation produced a beautiful background but subject need to be modified, to get that modification we will be forced to vary, maybe a little or a lot, the background for example, whereas with the chromakey system we could have no variation of the background and only variations of the subject.

If we input for example a subject on a greenscreen background as one layer image idea (t2i Clip Text generated or an i2i existing one) (with the subject devoid of green in its colors),
and then we pass the image to the encoder, the noise-denoise generated in the parts of the subject will be very variable but that generated in the green area will be very uniform in color especially since we specified to the text encoder that we want a full green matte background as a result behind the subject.
So when we move on to the decoding and sampler phase we could easily obtain a subject that can be cutout in his layer by chroma key technique (green is the example but can be any color).

This would result in an image of the subject isolated from the green screen background within the neural network process
not do after by humans... inside the workflow process!
Combined with the positioning techniques of ControlNet for posing, we can obtain in partial outputs all the separate alpha-keyed layers rendered to images (transparent Png).

At which point there could be a compositing node with standards mixing algorithms (alpha of course but too transparency, multiply, add , soft etc.) such as in blender, that creates a new final composite image but using fixed seeds for each layer image already cut out in key from the greenscreen generated one and for the background joints together, always also having all the images of the subjects and background layers separate which makes it possible for example after to change the position of a subject, the color grading of the background etc., as is normally done in 2D compositing and post render compositing in 3D.

I don't know if I explained it well, but Tell me what you think about it.

[ronruizG]

I think this a good idea but just at certain extend. I mean we have other "open source" tools for compositing so it's more important to have an output that could be easily fed to another tool, so I think any additional node should help to a easy workflow integration. So a green screen or +alpha_channel image would be great

[AceroMagico]

While waiting for the idea to be considered, I found a way to realise what I was saying with the existing nodes: workflow Two Subjects posing with ControlNet Depth + Bg, Full Layers Separations with Masks (key cuts made with VAEEncodeForInpaint produces better results than SetLatentNoiseMask although it is a little heavier in calculation), Vae, Lora epiNoiseOffset for better Composite Color Grading.(including masks if you want to try it and don't have any) workflow is in the "workflow Two Subjects Bg Full Layers Separation with Masks Vae Lora epiNoiseOffset Inpaint.png" 😉

[AceroMagico]

[WASasquatch]

I have added a MiDAS remove background node to WAS Suite that can easily do this for a lot of images. It outputs masked image and mask. Additionally I have color select where you can extract colors from a image to make a mask from.

#241

[AceroMagico]

I haven't had time to try it out yet, but from what I read your suite of nodes is fantastic, great!
This is exactly what I was saying and what was missing.
Your work takes ComfyUi to another level.

[WASasquatch]

I haven't had time to try it out yet, but from what I read your suite of nodes is fantastic, great! This is exactly what I was saying and what was missing. Your work takes ComfyUi to another level.

Well thank you 😌❤️