When loading Q5_K_M T5 with 11GB Fp8 Flux model, I get OOM sometimes

[Apacchi88]

When loading Q5_K_M T5 with 11GB Fp8 Flux model, I get OOM sometimes but not with T5 fp8 scaled safetensors.

For reference the T5 fp8 scaled safetensors is 4.8GB while T5 Q5_K_M is 3.15GB.

I have a 4060Ti 16GB and 64GB RAM.

It wasn't like this before.

Below is a video demonstrating this, first I load clip with GGUF, after it reaches 99% vram usage it either crashes or slows down a lot. Then I restart comfy I load the safetensors version and all is fine.

2024-12-05_17-47-39.1.mp4