Upscaler and TeaCache

[azrahello]

Hi, would it be possible to implement an upscaler? Either using ControlNet like this: https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Upscaler or something similar to https://github.com/ssitu/ComfyUI_UltimateSDUpscale? I find the ability to upscale images very useful. I was wondering if there’s a chance to implement it.
P.S. If you need a tester for your projects, I’m always available! 😉
Edit.

Yesterday, I tested this paper and this repository in ComfyUI using FLUX1 dev in 8-bit GGUF. By following this process, I was able to almost halve the number of iterations and the time required, effectively making it faster than MLX. I was wondering if this could be useful and if it could be implemented. It would be amazing to generate images with FLUX in almost half the traditional time!

[filipstrand]

Hi @azrahello! Upscaling is interesting for sure and should be something we should support! Thanks for linking the ControlNet one, I will look into this to get a sense of how much work it would take to implement... Depending on a few personal factors, I might be able to spend more time than typical on MFLUX over the coming time so I am really happy with your feature suggestions here :)

TeaCache looks awesome, definitely worth investigating! On their site, they have an example to get a sense of the quality degradation, and that looks really promising.

Would you say with your own testing that you got similary good results, or do they vary a lot? Anyway, really cool!

[azrahello]

this is the output of comfy without teacache,
Requested to load Flux
100%|███████████████████████████████████████████| 25/25 [01:17<00:00, 3.09s/it]
Prompt executed in 77.91 seconds
got prompt
this with teacache
Requested to load Flux
100%|███████████████████████████████████████████| 25/25 [00:40<00:00, 1.62s/it]
Prompt executed in 40.91 seconds this with teacache

this without teacache.

with a value of 0.40.
This is made with mflux.

time mflux-generate
--prompt "Luxury food photograph"
--model dev
--steps 25
--seed 1010
--height 512
--width 768
Fetching 8 files: 100%|████████████████████████| 8/8 [00:00<00:00, 75573.05it/s]
Fetching 7 files: 100%|████████████████████████| 7/7 [00:00<00:00, 19277.83it/s]
100%|███████████████████████████████████████████| 25/25 [01:02<00:00, 2.51s/it]
mflux-generate --prompt "Luxury food photograph" --model dev --steps 25 --see 3,65s user 39,06s system 64% cpu 1:06,27 total

[azrahello]

Hi! Still in the same field, I also found this node: Comfy-WaveSpeed, which in the README refers to this: ParaAttention. I’m not sure if they do the same thing, but while teacache works for me with GGUF, these others don’t.

Also, I saw that MLX Explore released this: “Support loading F8_E4M3 from safetensors”, which is a request I often see in nodes for loading the Flux model in ComfyUI (but it’s not usable because it requires too many resources). Maybe this could be useful for you? For me, it’s totally out of reach! :P

[filipstrand]

@azrahello Hi! Thanks for the pointers, I have not seen these before and your great suggestions are always welcome :)

I remember I tried the tea-cache technique a while back but did not give it enough time to properly evaluate it and put it on hold for now. It also felt a bit "hackish" with a few magic numbers in the implementation that made me a bit suspicious..., but can be worth revisiting.

For First Block Cache, it looks like they claim it is even simpler than TeaCache which sounds promising. I'll put this on the reading list and revisit after finishing up the 0.6.0 release!