You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
I have been testing models with llamafiler-0.8.17 (renamed llamafiler-0.8.17.exe) on Windows 11 (Powershell). It works fine with, for example, the following command/model: .\llamafiler-0.8.17.exe -m .\smollm2-1.7b-instruct-q4_k_m.gguf.
But when trying to load the embedding model specified in llamafiler help file (.\llamafiler-0.8.17.exe -m .\all-MiniLM-L6-v2.F32.gguf) I get the following error "failed to load model". Any ideas? Thanks. Seb
EDIT: the problem seems to be coming from the embedding model used. It finally worked with the following model: all-MiniLM-L6-v2.Q8_0.gguf. The other 2 models I had (all-MiniLM-L6-v2-q8_0.gguf and all-MiniLM-L6-v2-f32.gguf) with a slightly different spelling, were probably downloaded from the wrong repository on Hugging Face. My mistake.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi,
I have been testing models with llamafiler-0.8.17 (renamed llamafiler-0.8.17.exe) on Windows 11 (Powershell). It works fine with, for example, the following command/model:
.\llamafiler-0.8.17.exe -m .\smollm2-1.7b-instruct-q4_k_m.gguf
.But when trying to load the embedding model specified in llamafiler help file (
.\llamafiler-0.8.17.exe -m .\all-MiniLM-L6-v2.F32.gguf
) I get the following error "failed to load model". Any ideas? Thanks. SebEDIT: the problem seems to be coming from the embedding model used. It finally worked with the following model:
all-MiniLM-L6-v2.Q8_0.gguf
. The other 2 models I had (all-MiniLM-L6-v2-q8_0.gguf
andall-MiniLM-L6-v2-f32.gguf
) with a slightly different spelling, were probably downloaded from the wrong repository on Hugging Face. My mistake.Beta Was this translation helpful? Give feedback.
All reactions