CUDA out of memory - CUDA 12.3

[zzubrzycka]

Hey, is it possible to run this code for CUDA 12.3? I installed the right torch version for it. I keep getting this error about CUDA being out of memory. I tried setting max_split_size_mb to 64, but it didn't work. Do I need CUDA 11.8 to run this code? I have 8 GB VRAM, so I guess that's not an issue...?

[nitinmukesh]

Are you trying the sample image or your own image?
See the dimension of image is not too big

[zzubrzycka]

Are you trying the sample image or your own image? See the dimension of image is not too big

I am trying first on provided sample pictures, I checked their dimensions

[nitinmukesh]

Post a screen shot of your task manager and log from command prompt

[zzubrzycka]

Post a screen shot of your task manager and log from command prompt

I'm working on linux, so I'm not sure how much shared memory I have, I just know about 8 GB VRAM (if I can provide something better, please tell me how). I installed also cuda 11.8, but I still have the cuda out of memory error. I tried to set:
"export PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128",
then all 20/20 steps go smoothly and right after that I have the cuda out of memory error. When I set it to 64mb, I have other errors:
"RuntimeError: cusolver error: CUSOLVER_STATUS_INTERNAL_ERROR, when calling cusolverDnCreate(handle)"

Everything works fine when I implemented it in Google Colab. And here only the mask saves, the actual output obviously not. I already set the sample size to 1 instead of 4.

Is it possible to do anything? Or do I just have too little GPU memory? I'm using NVIDIA GeForce RTX 3050

[zzubrzycka]

Post a screen shot of your task manager and log from command prompt

It started working when I killed the process 869572, but only sometimes. I really do not know why, but sometimes it doesn't work even with the same exact pictures that it just have worked with. I use "watch nvidia-smi" all the time and see that sometimes it uses even less memory when it doesn't work than when it works.
Can I do something to for example have worse resolution or something, but for it to work each time? Usually my GPU memory lacks around 20 MiB, which is super little for it to be a problem not to solve

[SubGlitch1]

Post a screen shot of your task manager and log from command prompt

It started working when I killed the process 869572, but only sometimes. I really do not know why, but sometimes it doesn't work even with the same exact pictures that it just have worked with. I use "watch nvidia-smi" all the time and see that sometimes it uses even less memory when it doesn't work than when it works. Can I do something to for example have worse resolution or something, but for it to work each time? Usually my GPU memory lacks around 20 MiB, which is super little for it to be a problem not to solve

interestingly enough i have the same issue but with 12gb of vram and unlike yours it doesnt work whatsoever i kill. What was the process you killed and what cli options did you use?

[nitinmukesh]

Unfortunately I can't help with issues on Linux. :(

[SubGlitch1]

Unfortunately I can't help with issues on Linux. :(

do you know how much VRAM this roughly takes up?