Inpaint Model is unloaded despite --always-gpu --disable-offload-from-vram #316
Comments
|
remove |
Thanks for the quick reply, unfortunately that made it even worse, now taking over 50 seconds for the request to complete. Moving the models itself takes around 35 seconds now. |
|
Sorry. I did more testing based on the information you provided, although not very sure, but I think it may have been due to insufficient available GPU memory. Due to equipment problems, I can only do limited testing here. If you can, can you please test how well Fooocus-API runs when it has exclusive GPUs |
|
Can you clarify what you mean by exclusive GPUs? |
|
I want to mention also that after testing, the same issue happens with the regular Fooocus too, not just with using your API. |
It means do not run any other programs that occupy GPU memory。 |
Got it, yes im not doing that. I run only Fooocus on a RTX A6000 Ada with 48GB of vram |
|
Okay so as far as i understand it, when patching the model for inpainting, it creates a clone. Clones are always unloaded afterwards. Even if i remove the code for unloading the clones, when it looks for already loaded models, the new clone doesnt exactly match the old one, so it is reloaded regardless. Any idea how to fix this? |
It seems that you have studied the Fooocus code more deeply than I have, and I have seen #2811 I will review the code and try fix it |
|
Thanks for looking into this, im not too fluent in python unfortunately. C# & Dart/Flutter are my main languages. If i can test anything or if you have anything specific i can look into, let me know, im available. |
|
I did some more testing. Changing the code i mentioned in 2811 to this quick workaround:
works, it finds the model. HOWEVER: The model is still a clone, so it is being unloaded. I can prevent the unloading of clones by just commenting out the unloading in the unload_model_clones method, then it actually works, it doesnt reload the model. Also the new clone is still being created, which fills up the vram. BUT: then i get some artifacting in the image. I have attached an image without this modification and with the modification. It almost seems like somehow the refiner is not run for the last few steps but im not 100% sure.
|
|
Find this issue with google. Any update @mrhan1993 @user425846 ? Willing to pay you for fixing |
|
I dont have a solution yet, maybe mrhan has one |
I am trying, but not solve yet |
|
Let me know if i can do anything to help, im very eager to fix this |
|
@mrhan1993 any update ? |
Haha i think we are stressing him too much :D |
yes 😅 |
|
I carefully tracked the whole generation process, and I'm not sure if this is the cause of the problem. That is, the SDXL model is always unloaded and then reloaded, while other models just go from one list to another. The reason for reloading is that the line I compared its changes in processing, and there are some differences between SDXL in
I have already tried the same seed, it's not work I will continue to explore this question, but I am not sure if I can finally find out what the problem is. |
|
Willing to pay you 300$ btc for fixing @mrhan1993 |
|
One likely pessimistic expectation is that this may not be a BUG. During the processing of Inipaint, information is added to the SDXL model, that is, the |
|
Hi @mrhan1993 @mashb1t found the solution. It is just a single line that needs to be changed, you can find it in lllyasviel/Fooocus#2811 Line 362 in async_worker.py The issue is, that while this works on a clean install of Fooocus, it does not work with your API, if i edit the line in Fooocus-API/repositories/Fooocus/modules/async_worker.py it is not working. Maybe with this information, you would be able to fix this and maybe even add it as a setting in config.txt or as a request parameter? As there are some things that do not work with this change:
|
|
@user425846 Try this in |
|
Just tried it out, setting |
This is really bad news 😂 |
|
Is it a bigger problem than it seems like? Because the original solution for the standard Fooocus install is very simple, just this one small change. |
|
😂, This is the problem. |
|
I do a standard install for Fooocus, and change and then execute inpaint twice with the default parameters and background prompt Here are my run log and Fooocus page parameters. Can you provide your success log and Fooocus page options for success and all the other things you can think of?
|
|
You wont believe me but im unable to reproduce my success from yesterday. Setting it to false does not fix it anymore. Man, i didnt expect this problem to be so much work hahaha |
|
I have tried it multiple times now on fresh installations and i cant get it to work anymore. Im lost right now. |
😂 😂 , may be your success is txt2img |
@user425846 I'm sorry and also clueless, can't support / debug any further as it works on my machine(s), both Macbook and Windows... Can you please confirm it still creating clones when using the adjusted code in Colab? |
I have tested it again right now and i "solved" it by choosing a faster server to run it on, so it is much quicker to load the model. I have attached the full log below, you can see that it is still unloading clones and moving models at every request. You can also see it correctly isnt using any refiner and even logs
|
Hi,
despite using the arguments, every time when an inpaint request is started, it is moving models. This takes over 10 seconds every time. I have tried disabling all loras and styles, but that didnt change anything. Any ideas? This is not usable like that for me unfortunately. Below is the log for launching and doing 2x identical inpainting requests back to back.
The text was updated successfully, but these errors were encountered: