New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Possible memory leak? #6

Open

oursland opened this issue Nov 18, 2024 · 1 comment

Contributor

oursland commented Nov 18, 2024

When running the application, a single query can result in 133 GiB of RAM being allocated before it fails.

My system is a MBP M2 Max with 96 GiB of RAM, so much of this is coming out of swap space.

Contributor Author

oursland commented Nov 18, 2024

I have had success loading the model in LM-Studio as a LLaMA 3.1 model. The memory consumption in LM-Studio is stable at 31 GiB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment