Skip to content

Commit

Permalink
llama : use ggml_backend_dev_get_extra_bufts
Browse files Browse the repository at this point in the history
This commit updates the `name` argument for the
`ggml_backend_reg_get_proc_address` function call in
`make_cpu_buft_list` to be `ggml_backend_dev_get_extra_bufts`
instead of `ggml_backend_cpu_get_extra_bufts`.

The motivation for this change is that in
`ggml_backend_cpu_get_proc_address`, the `name` parameter is matched against
`ggml_backend_dev_get_extra_bufts` and not
`ggml_backend_cpu_get_extra_bufts`:
```c++
static void * ggml_backend_cpu_get_proc_address(ggml_backend_reg_t reg,
    const char * name) {
    if (strcmp(name, "ggml_backend_set_n_threads") == 0) {
        return (void *)ggml_backend_cpu_set_n_threads;
    }
    if (strcmp(name, "ggml_backend_dev_get_extra_bufts") == 0) {
        return (void *)ggml_backend_cpu_get_extra_bufts;
    }

    return NULL;

    GGML_UNUSED(reg);
}
```
  • Loading branch information
danbev committed Nov 9, 2024
1 parent ec450d3 commit a1a7ab0
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/llama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7189,7 +7189,7 @@ static llama_model::buft_list_t make_cpu_buft_list(llama_model & model) {
auto * cpu_dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_CPU);
auto * cpu_reg = ggml_backend_dev_backend_reg(cpu_dev);
auto ggml_backend_dev_get_extra_bufts_fn = (ggml_backend_dev_get_extra_bufts_t)
ggml_backend_reg_get_proc_address(cpu_reg, "ggml_backend_cpu_get_extra_bufts");
ggml_backend_reg_get_proc_address(cpu_reg, "ggml_backend_dev_get_extra_bufts");
if (ggml_backend_dev_get_extra_bufts_fn) {
ggml_backend_buffer_type_t * extra_bufts = ggml_backend_dev_get_extra_bufts_fn(cpu_dev);
while (extra_bufts && *extra_bufts) {
Expand Down

0 comments on commit a1a7ab0

Please sign in to comment.