How to reduce condition graph computing time?

[hxgqh]

it consumes 34 seconds, while sampling only consumes 5.5s/it

[Green-Sky]

flux employs t5xxl, which is relatively heavy. And it is only implemented to run on cpu right now.

[stduhpf]

@hxgqh Set the numbers of threads with the -t argument.
With a single thread it also takes around 34 seconds on my CPU.
With -t 24 (I have a 24 thread CPU) , it only takes around 3.5 seconds

[hxgqh]

@stduhpf I changed threads to be 2/4/8/24/48/96, and it seems not working. Maybe the default threads is set to be cores of cpu? #3

[stduhpf]

You're right @hxgqh , it uses the number of physical cores of the CPU by default. (so 12 in my case)
If I don't set the -t argument at all, it takes 4 seconds with 12 threads.

Then I guess either your CPU is too slow, or you're running out of system memory and it's using swap (if that's the case, maybe using a quantized version of t5xxxl could help).

[hxgqh]

@Green-Sky Is there any plan to run t5xxl on GPU?

[actionless]

it seems smth is off with it, that same t5xxl step on ryzen 5600x the python cpu implementation (from diffusers lib) running 67 seconds faster (237%) than CPP implementation

and that's actually the best of sd-cpp measurements, with 6 threads, when setting to 12 (nproc) - it becomes 22 seconds even slower

• python diffusers (cpu) - 52 seconds
• sd-cpp 3 threads - 199 seconds
• sd-cpp 6 threads - 119 seconds
• sd-cpp 12 (nproc) threads - 141 seconds
• sd-cpp 16 threads - 147 seconds

offtopic:
inference step takes ~9 seconds for both diffusers gpu and sd-cpp gpu, however for achieving the same quality of result diffusers taking 4 steps, and sd-cpp needs 20 - but i guess the latter is smth to do with the default parameters

[actionless]

i found the solution for my problem with high condition graph computing time, mb would be useful for someone else:

because i was building the package on old xeon server, but running on a new ryzen workstation - apparently some cpu optimizations were disabled during compilation

so after re-compiling it on the workstation itself - condition graph computes now in 18 seconds (34 seconds faster than diffusers-cpu and 100 seconds faster than before, without those optimizations) in the same testcase