The most comprehensive GUI tool for GGUF model quantization. Stop wrestling with command lines - quantize, merge, and optimize your models with just a few clicks.
- π© Download and manage llama.cpp backends
- ποΈ Select and quantize GGUF models
- π Configure quantization parameters
- π» Monitor system resources during quantization
- β³ Parallel quantization + imatrix generation
- π LoRA conversion and merging
- π Preset saving and loading
- 8οΈβ£ AutoFP8 quantization
- πͺ GGUF splitting and merging
- π HTTP API for automated monitoring
- Fast: Saves time on manual configuration
- Simple: Clean UI, no terminal needed
- Powerful: Handles models up to infinite size, only limited by your RAM
- Resource-aware: Optimized memory management and efficient UI library
git clone https://github.com/leafspark/AutoGGUF
cd AutoGGUF
- Install dependencies:
pip install -r requirements.txt
- Run the application:
or use the
python src/main.py
run.bat
script.
macOS and Ubuntu builds are provided with GitHub Actions, you may download the binaries in the releases section.
Standard builds:
- Download the latest release
- Extract all files to a folder
- Run
AutoGGUF-x64.exe
- Any necessary folders will be automatically created
Setup builds:
- Download setup variant of latest release
- Extract all files to a folder
- Run the setup program
- The .GGUF extension will be registered with the program automatically
- Run the program from the Start Menu or desktop shortcuts
After launching the program, you may access its local server at port 7001 (set AUTOGGUF_SERVER
to "enabled" first)
gpg --import AutoGGUF-v1.5.0-prerel.asc
gpg --verify AutoGGUF-v1.9.1-Windows-avx2.zip.sig AutoGGUF-v1.9.1-Windows-avx2.zip
sha256sum -c AutoGGUF-v1.9.1.sha256
# Import the public key
gpg --import AutoGGUF-v1.5.0-prerel.asc
# Verify the signature
gpg --verify AutoGGUF-v1.9.1-Windows-avx2.zip.sig AutoGGUF-v1.9.1-Windows-avx2.zip
# Check SHA256
$fileHash = (Get-FileHash -Algorithm SHA256 AutoGGUF-v1.9.1-Windows-avx2.zip).Hash.ToLower()
$storedHash = (Get-Content AutoGGUF-v1.9.1.sha256 | Select-String AutoGGUF-v1.9.1-Windows-avx2.zip).Line.Split()[0]
if ($fileHash -eq $storedHash) { "SHA256 Match" } else { "SHA256 Mismatch" }
Release keys are identical to ones used for commiting.
pip install -U pyinstaller
./build.sh RELEASE | DEV
cd build/<type>/dist/
./AutoGGUF
pip install -U pyinstaller
build RELEASE | DEV
Find the executable in build/<type>/dist/AutoGGUF.exe
.
You can also use Nuitka, which may result in a slower build but a faster output executable:
build_optimized RELEASE | DEV
View the list of supported languages at AutoGGUF/wiki/Installation#configuration (LLM translated, except for English).
More languages will be updated as soon as possible!
To use a specific language, set the AUTOGGUF_LANGUAGE
environment variable to one of the listed language codes (note: some languages may not be fully supported yet, those will fall back to English).
- Some inconsistent logging
- Time estimation for quantization
- Quantization file size estimate
- Perplexity testing
- bitsandbytes (coming soon)
- SSL module cannot be found error: Install OpenSSL or run from source using
python src/main.py
with therun.bat
script (pip install requests
) - Check out the Wiki for advanced usage
Fork the repo, make your changes, and ensure you have the latest commits when merging. Include a changelog of new features in your pull request description. Read CONTRIBUTING.md
for more information.