error: failed to open models/7B/ggml-model-f16.gguf: No such file or directory

[aszenz]

I get this error on running the project on a NixOS machine with no GPU, does this need an NVIDIA gpu to run?

1655000K .......... .......... .......... .......... .......... 39% 21.8M 1m48s
1655050K .......... .......... .......... .......... .......... 39% 24.3M 1m48s
1655100K .......... .......... .......... .......... .......... 39% 24.6M 1m48s
1655150K .......... .......... .......... .......... .......... 39% 24.3M 1m48s
1655200K .......... .......... .......... .......... .......... 39% 24.2M 1m48s
1655250K .......... .......... .......... .......... .......... 39% 24.6M 1m48s
1655300K .......... .......... .......... .......... .......... 39% 24.1M 1m48s
1655350K .......... .......... .......... .......... .......... 39% 18.0M 1m48s
1655400K .......... .......... .......... .......... .......... 39% 24.3M 1m48s
1655450K .......... .......... .......... .......... .......... 39% 24.6M 1m48s
1655500K .......... .......... .......... .......... .......... 39% 22.5M 1m48s
1655550K .......... .......... .......... .......... .......... 39% 20.2M 1m48s
1655600K .......warning: couldn't find nvcc (nvidia c compiler) try setting $CUDA_PATH if it's installed
{"timestamp":1701335522,"level":"INFO","function":"main","line":2258,"message":"build info","build":1500,"commit":"a30b324"}
error: failed to open models/7B/ggml-model-f16.gguf: No such file or directory
{"timestamp":1701335522,"level":"INFO","function":"main","line":2261,"message":"system info","n_threads":4,"n_threads_batch":-1,"total_threads":8,"system_info":"AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | "}
warning: couldn't find nvcc (nvidia c compiler) try setting $CUDA_PATH if it's installed
{"timestamp":1701335572,"level":"INFO","function":"main","line":2258,"message":"build info","build":1500,"commit":"a30b324"}
{"timestamp":1701335572,"level":"INFO","function":"main","line":2261,"message":"system info","n_threads":4,"n_threads_batch":-1,"total_threads":8,"system_info":"AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | "}
error: failed to open models/7B/ggml-model-f16.gguf: No such file or directory

• Terminal will be reused by tasks, press any key to close it.

[tluyben]

No you don't need a GPU to run; it's made for CPU. I have seen this issue before on and it seems to have to do something with permissions. Can you see if the file is actually there or not, because for other people who reported it seems the file is there but from inside docker it's not accessible.