Implementing RedPajama models

[georgepullen]

Hello, Is there a way I can replace the main.exe file (that is used for inference with LLaMa when you run ./main) with the redpajama.exe file?

[gjmulder]

That's not how it works. main is provided by llama.cpp. llama-cpp-python provides a Python interface to llama.cpp.

HuggingFace provides access to a wide range of GGML models that are mostly compatible with llama.cpp and therefore llama-cpp-python.

Some of these models were trained using open source datasets such as Red Pajama. Most are based on Facebook's Llama models and are therefore distributed under a restricted license.

[georgepullen]

Sorry, I should clarify I'm referring to Together Computer's INCITE 3b and 7b models.

In regards to the redpajama.exe file, i'm referring to the one in the redpajama.cpp fork of llama.cpp. Since the redpajama.exe file works as a drop in replacement for the main.exe file, I was wondering if I could somehow replace it in llama-cpp-python in order to get python bindings working for redpajama.cpp.