Skip to content

Commit

Permalink
common : add llama.vim preset for Qwen2.5Coder1.5B
Browse files Browse the repository at this point in the history
This commit adds a preset for llama.vim to use the default Qwen 2.5
Coder 1.5B model.

The motivation for this change is to make it easier to start a server
suitable to be used with the llama.vim plugin. For example, the server
can be started with the following command:
```console
$ llama.vim --vim-qwen2.5-Coder-1.5B-default
```

Refs: ggml-org#10932
  • Loading branch information
danbev committed Feb 18, 2025
1 parent 63ac128 commit ab1728d
Showing 1 changed file with 16 additions and 0 deletions.
16 changes: 16 additions & 0 deletions common/arg.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2501,5 +2501,21 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
}
).set_examples({LLAMA_EXAMPLE_EMBEDDING, LLAMA_EXAMPLE_SERVER}));

add_opt(common_arg(
{"--vim-qwen2.5-Coder-1.5B-default"},
string_format("use default Qwen 2.5 Coder 1.5B (note: can download weights from the internet)"),
[](common_params & params) {
params.hf_repo = "ggml-org/Qwen2.5-Coder-1.5B-Q8_0-GGUF";
params.hf_file = "qwen2.5-coder-1.5b-q8_0.gguf";
params.port = 8012;
params.n_gpu_layers = 99;
params.flash_attn = true;
params.n_ubatch = 1024;
params.n_batch = 1024;
params.n_ctx = 0;
params.n_cache_reuse = 256;
}
).set_examples({LLAMA_EXAMPLE_SERVER}));

return ctx_arg;
}

0 comments on commit ab1728d

Please sign in to comment.