Support applying FluxMod to GGUF models #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
Set Flux memory usage factor
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fox courtesy of Q5_K_S Flux + FluxMod with lite patch.
This pull adds simple GGUF support if ComfyUI-GGUF is available. (I say simple because the advanced GGUF loader allows setting stuff like the dequant dtype:
It wouldn't be hard to allow setting those parameters with FluxMod but you'd either need GGUF-specific FluxMod loader nodes or to add GGUF fields to the base nodes which might not apply.
I also disabled the casting layers and channels last stuff for GGUF models, scared of messing with GGUF format weights. I don't know for sure if that was necessary.
I'm pretty sure this shouldn't break any non-GGUF models (seems fine in my testing) and it seems to work for the GGUF model I tested with and without lite patch. Definitely not sure that the GGUF support is optimal but at least this will enable basic use.
This also includes a few other fixes/cleanups: The copy of
math.pyfrom Comfy was unneeded, the only difference was having a mask argument, but you aren't passing a mask in the actual model layers (not sure if that's intentional) so it makes no difference. I also set the memory usage factor in the model to the normal Flux value. The default inBASEseemed to be too low - it was easy to run out of memory for stuff like previews since ComfyUI wasn't estimating the memory usage correctly.BASEdefaults to 2.0, Flux uses 2.8. It possibly could be reduced somewhat, but I am not sure it's actually related to the model parameters or just the max amount of memory one operation like attention would require.