Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix support for Metal on macOS systems #1

Open
BrutalCoding opened this issue Dec 5, 2023 · 1 comment
Open

Fix support for Metal on macOS systems #1

BrutalCoding opened this issue Dec 5, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@BrutalCoding
Copy link
Owner

By default, llama.cpp uses Metal GPU acceleration when running on macOS.

This works fine when I used llama.cpp on its own, but things get complicated when trying to bundle this in a native Mac app.

When bundling the native dynamic library for macOS, and leaving Metal support turned on, I would get the following error when compiling the macOS project:

"ggml_metal_init: found device: Apple M1 Pro
ggml_metal_init: picking default device: Apple M1 Pro
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd
ggml_metal_init: loading 'ggml-metal.metal'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file “ggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x1572d8b70 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
llama_new_context_with_model: ggml_metal_init() failed"

I've tried bundling the .metal file within the native macOS project directory, in various locations, to no avail. I do think I can solve this, I just haven't gotten the time to spend more hours on this issue.

As for the time being, the current macOS library doesn't make use of Metal and instead relies on CPU entirely.

@BrutalCoding BrutalCoding added the enhancement New feature or request label Dec 5, 2023
@BrutalCoding BrutalCoding self-assigned this Dec 5, 2023
@jianhuihi
Copy link

any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants