You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
By default, llama.cpp uses Metal GPU acceleration when running on macOS.
This works fine when I used llama.cpp on its own, but things get complicated when trying to bundle this in a native Mac app.
When bundling the native dynamic library for macOS, and leaving Metal support turned on, I would get the following error when compiling the macOS project:
"ggml_metal_init: found device: Apple M1 Pro
ggml_metal_init: picking default device: Apple M1 Pro
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd
ggml_metal_init: loading 'ggml-metal.metal'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file “ggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x1572d8b70 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
llama_new_context_with_model: ggml_metal_init() failed"
I've tried bundling the .metal file within the native macOS project directory, in various locations, to no avail. I do think I can solve this, I just haven't gotten the time to spend more hours on this issue.
As for the time being, the current macOS library doesn't make use of Metal and instead relies on CPU entirely.
The text was updated successfully, but these errors were encountered:
By default, llama.cpp uses Metal GPU acceleration when running on macOS.
This works fine when I used llama.cpp on its own, but things get complicated when trying to bundle this in a native Mac app.
When bundling the native dynamic library for macOS, and leaving Metal support turned on, I would get the following error when compiling the macOS project:
I've tried bundling the .metal file within the native macOS project directory, in various locations, to no avail. I do think I can solve this, I just haven't gotten the time to spend more hours on this issue.
As for the time being, the current macOS library doesn't make use of Metal and instead relies on CPU entirely.
The text was updated successfully, but these errors were encountered: