GGUF.js

Download, manage, and run Llama GGUF files easily with Backyard.ai

A Javascript library (with Typescript types) to parse metadata for GGML-based GGUF files.

Supported Architectures

• Llama
• MPT
• GPTNeoX
• GPTJ
• GPT2
• Bloom
• Falcon
• RWKV
• Gemma

This library goal is to be 1-to-1 with the spec. PRs welcome!

Install

yarn add gguf

or

npm install gguf

Usage

import gguf, { isLLamaMetadata } from 'gguf'

// pass in a file path, gguf.js will only load in what is needed for the metadata
// not the whole file
const { metadata, error } = await gguf('./llama2.gguf')

if (error) {
  throw error
}

// helper function to give full type safety
// see more in `src/index.ts`
if (isLLamaMetadata(metadata)) {
  console.log(`context length: ${metadata.llama.context_length}`)
}

TypeScript

Typescript is supported by default and all definitions are in src/metadataTypes

License

MIT © Ahoy Labs, Inc.