Skip to content
This repository has been archived by the owner on Jan 31, 2024. It is now read-only.

Latest commit

 

History

History
80 lines (43 loc) · 4.1 KB

README.md

File metadata and controls

80 lines (43 loc) · 4.1 KB
Raw

GitHub last commit (by committer) LLamaStack Badge Web Demo Web API Demo Discord

Welcome to LLamaStack!

LLamaStack is a library that provides higher-level services and integrations for .NET applications, enhancing the functionality and versatility of the LLamaSharp and llama.cpp projects.

This repository also contains a collection of barebone/bootstrap UI & API projects. Here, you'll find various user interface applications, including Web, API, WPF, and Websocket, all built to showcase the capabilities of the LLamaStack.

Overview

LLamaStack is built on top of the popular LLamaSharp and llama.cpp projects, extending their functionalities with a range of user-friendly UI applications. LLamaSharp is a powerful library that provides C# interfaces and abstractions for the popular llama.cpp, the C++ counterpart that offers high-performance inference capabilities on low end hardware. LLamaStack complements these projects by creating intuitive UI & API interfaces, making the power of LLamaSharp and llama.cpp more accessible to users.

Projects

LLamaStack is a comprehensive library with several projects tailored for different purposes:

  1. LLamaStack.Core: This project offers high-level services and integrations for .NET applications.

  2. LLamaStack.Web: The ASP.NET Core Web interface provides all the core functions of llama.cpp & LLamaSharp.

  3. LLamaStack.WPF: The WPF UI interface provides all the core functions of llama.cpp & LLamaSharp.

  4. LLamaStack.WebAPI: This is an implementation of an ASP.NET Core WebAPI with all the essential features of llama.cpp & LLamaSharp.

In Development

  1. LLamaStack.Signalr: SignalR websocket server and client implementations designed for use in web and .NET environments.

  2. LLamaStack.SemanticKernel: Support for Microsoft.SemanticKernel using local models and the LLamaStack.WebAPI and LLamaStack.Signalr implementations

Installation

LLamaStack can be found via the nuget package manager, download and install it.

PM> Install-Package LLamaStack

LLamaStack relies on the llama.cpp and LLamaSharp libraries.

  • LLamaSharp is conveniently included in the NuGet package.
  • However, you will need to obtain llama.cpp separately. You can either download or compile it yourself.

Alternatively, you can opt for one of the LLamaSharp backend Nuget packages tailored for your specific system.

LLamaSharp.Backend.Cpu  # CPU for Windows, Linux and Mac
LLamaSharp.Backend.Cuda11  # GPU CUDA11 for Windows and Linux
LLamaSharp.Backend.Cuda12  # GPU CUDA12 for Windows and Linux
LLamaSharp.Backend.MacMetal  # GPU Metal for Mac

Getting Started

To start using a specific UI project, please refer to the README file located in its respective directory.

You can store model configurations in the appsettings.json file. For examples and documentation on the appsettings.json structure, please see the appsettings.json Documentation.

LLamaStack also provides functionality for adding custom sections and loading/saving at runtime.

Contribution

We welcome contributions to LLamaStack! If you have any ideas, bug reports, or improvements, feel free to open an issue or submit a pull request.