common: llama_load_model_from_url using --model-url

[phymbert]

Motivation

Since GGUF is officially supported on huggingface it can be useful to start the cli directly from a http url.

We considered this at some point #4735, but it opens a lot of security concerns due to the core library having to execute commands.
So we settled with #5501 which should be good enough for most cases, but when there is no shell available it cannot work (example docker distroless container).

Changes

• llama_load_model_from_url to first download the file if it does not exist locally or the remote is newer, then call llama_load_model_from_file in common
• introduce --model-url which will trigger the download to --model in llama_init_from_gpt_params
• enable this feature if -DLLAMA_CURL=ON in cmake and make toolchains
• server tests using this parameter in embeddings.feature

Attention points

• The common will be dynamic linked to libcurl.
• It can later be moved to llama API if there is positive feedback from the community.
• it saved ${model_path}.etag or ${model_path}.lastModified files along with the model_path
• It's not possible to pass a custom CA, system CAs are always used in this first implementation on both posix and windows platform

Task

• download is triggered only if the file has changed based on etag and last-modified http headers
• CI build is passing
• Server CI build is passing

References

• llama : auto download HF models if URL provided #4735
• scripts : add hf.sh helper script #5501
• Closes Download model gguf file from hf #6085

[phymbert]

@ggerganov Georgi, if you approve the proposal with libcurl dependency I can continue further to support Etag headers to avoid downloading the file each time.

[ggerganov]

Yes, this can work. The curl dependency is optional, the implementation is isolated in common. Looks great

[Artefact2]

Supporting split GGUF files (for models above 50GB) would be nice too. Ideally you'd merge files with copy_file_range() when available to avoid extra disk I/O.

[phymbert]

Supporting split GGUF files (for models above 50GB) would be nice too. Ideally you'd merge files with copy_file_range() when available to avoid extra disk I/O.

Yes I am working on this in a separate branch:

• GGUF huge files size / Split possible ? #6000

[phymbert]

Someone can help with the error in the windows build ?

D:\a\llama.cpp\llama.cpp\common\common.cpp(906,52): error C1061: compiler limit: blocks nested too deeply [D:\a\llama.cpp\llama.cpp\build\common\common.vcxproj]

Also @ggerganov please double check my Makefile changes as I am not used to.

[dranger003]

Someone can help with the error in the windows build ?

D:\a\llama.cpp\llama.cpp\common\common.cpp(906,52): error C1061: compiler limit: blocks nested too deeply [D:\a\llama.cpp\llama.cpp\build\common\common.vcxproj]

Also @ggerganov please double check my Makefile changes as I am not used to.

I submitted a PR #6101 to fix this issue.
If you are in a hurry, you can try this workaround #6096

[phymbert]

I submitted a PR #6101 to fix this issue. If you are in a hurry, you can try this workaround #6096

Thanks a lot, no let's wait for your PR to be merged first.

[phymbert]

Need to fix the windows server CI tests, it's not passing at the moment

[phymbert]

It finally works on both windows and linux CI tests. Happy to merge it when CI passes

[phymbert]

@ggerganov Georgi, I suffered a little on Windows, but this implementation works on both platform. Would you please do another review as I changed a little bit the logic - and sorry but still need to find a good linter for CLion.

[cebtenzzre]

@phymbert This calls TerminateProcess(handle, SIGTERM) on Windows and os.kill(pid, signal.SIGKILL) on Unix. os.kill(pid, signal.SIGTERM) also calls TerminateProcess on Windows. psutil really seems like overkill here.

[phymbert]

Again,feel free to issue a PR