how about make load diffusion model+ support gguf

[xueqing0622]

how about make load diffusion model+ support gguf
there are example load model include gguf,
https://github.com/blepping/ComfyUI_FluxMod/tree/feat_gguf
lodestone-rock/ComfyUI_FluxMod#15

[blepping]

I'm not sure it would make sense. The difference between the loaders is being able to set some extra dtypes. The advanced GGUF loader lets you enter the dequant and patch dtypes:

You can't enter something like FP8 there, but I'm not sure how that would work. Dequant to fast FP8 might be possible, but dequantizing to another quantized format is probably going to be a big quality loss (and likely would need support implemented in ComfyUI-GGUF). It also would only work on GPUs that support the FP8 ops with ComfyUI's --fast parameter.

[chengzeyi]

Do you mean converting gguf models to native format so that it could work with ComfyUI better?

[xueqing0622]

I'm using gguf which works just like normal safetensors, and it seems that the threshold has a lot to do with the number of steps,
#19 ,
this morning I've heard from videos that the load diffusion model+ has better support for lora. So I asked if load diffusion model+ is ready to support gguf.

[chengzeyi]

I'm using gguf which works just like normal safetensors, and it seems that the threshold has a lot to do with the number of steps,

#19 ,

this morning I've heard from videos that the load diffusion model+ has better support for lora. So I asked if load diffusion model+ is ready to support gguf.

yes support for Lora should be better since it fuses the Lora into the weights which makes the inference faster