Python bindings with Hugging Face Hub and LangChain support

[marella]

Hi,

I created Python bindings for the GGML models: https://github.com/marella/ctransformers

Currently it supports GPT-2, GPT-J, GPT-NeoX, Dolly V2, StarCoder from the examples. Will continue to add more models.

It provides a unified interface for all models:

from ctransformers import AutoModelForCausalLM

llm = AutoModelForCausalLM.from_pretrained('/path/to/ggml-gpt-2.bin', model_type='gpt2')

print(llm('AI is going to'))

It can be used with models hosted on the Hugging Face Hub:

llm = AutoModelForCausalLM.from_pretrained('marella/gpt-2-ggml')

It has built-in support for LangChain:

from ctransformers.langchain import CTransformers

llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2')

print(llm('AI is going to'))

Please see README for more details.