diff --git a/README.md b/README.md index efac53f..454dd87 100644 --- a/README.md +++ b/README.md @@ -2,30 +2,49 @@ # AutoGGUF - automated GGUF model quantizer -AutoGGUF provides a graphical user interface for quantizing GGUF models -using the llama.cpp library. It allows users to download different versions of -llama.cpp, manage multiple backends, and perform quantization tasks with various -options. - -## Features: -1. Download and manage llama.cpp backends -2. Select and quantize GGUF models -3. Configure quantization parameters -4. Monitor system resources during quantization - -## Usage: - -**Cross platform**: - 1. Install dependencies, either using the `requirements.txt` file or `pip install PyQt6 requests psutil`. - 2. Run the `run.bat` script to start the application, or run the command `python src/main.py`. - -**Windows**: - 1. Download latest release, extract all to folder and run `AutoGGUF.exe` - 2. Enjoy! - -## Building: - -**Cross platform**: +[![Powered by llama.cpp](https://img.shields.io/badge/Powered%20by-llama.cpp-green.svg)](https://github.com/ggerganov/llama.cpp) +![GitHub release](https://img.shields.io/github/release/leafspark/AutoGGUF.svg) +![GitHub last commit](https://img.shields.io/github/last-commit/leafspark/AutoGGUF.svg) +![GitHub stars](https://img.shields.io/github/stars/leafspark/AutoGGUF.svg) +![GitHub forks](https://img.shields.io/github/forks/leafspark/AutoGGUF.svg) +![GitHub top language](https://img.shields.io/github/languages/top/leafspark/AutoGGUF.svg) +![GitHub repo size](https://img.shields.io/github/repo-size/leafspark/AutoGGUF.svg) +![GitHub license](https://img.shields.io/github/license/leafspark/AutoGGUF.svg) + +AutoGGUF provides a graphical user interface for quantizing GGUF models using the llama.cpp library. It allows users to download different versions of llama.cpp, manage multiple backends, and perform quantization tasks with various options. + +## Features + +- Download and manage llama.cpp backends +- Select and quantize GGUF models +- Configure quantization parameters +- Monitor system resources during quantization + +## Usage + +### Cross-platform +1. Install dependencies: + ``` + pip install -r requirements.txt + ``` + or + ``` + pip install PyQt6 requests psutil shutil + ``` +2. Run the application: + ``` + python src/main.py + ``` + or use the `run.bat` script. + +### Windows +1. Download the latest release +2. Extract all files to a folder +3. Run `AutoGGUF.exe` + +## Building + +### Cross-platform ```bash cd src pip install -U pyinstaller @@ -33,56 +52,55 @@ pyinstaller main.py --onefile cd dist/main ./main ``` -**Windows**: + +### Windows ```bash build RELEASE/DEV ``` -Find exe in `build//dist/AutoGGUF.exe`. +Find the executable in `build//dist/AutoGGUF.exe`. + +## Dependencies -## Dependencies: - PyQt6 - requests - psutil - shutil - OpenSSL -## Localizations: +## Localizations -View the list of languages supported at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English) +View the list of supported languages at [AutoGGUF/wiki/Installation#configuration](https://github.com/leafspark/AutoGGUF/wiki/Installation#configuration) (LLM translated, except for English). -In order to use them, please set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes. +To use a specific language, set the `AUTOGGUF_LANGUAGE` environment variable to one of the listed language codes. + +## Known Issues -## Issues: - Saving preset while quantizing causes UI thread crash (planned fix: remove this feature) -- Cannot delete task while processing, you must cancel it first or the program crashes (planned fix: don't allow deletion before cancelling, or cancel automatically) +- Cannot delete task while processing (planned fix: disallow deletion before cancelling or cancel automatically) - Base Model text still shows when GGML is selected as LoRA type (fix: include text in show/hide Qt layout) -- ~~Cannot disable llama.cpp update check on startup~~ (fixed in v1.3.1) -- ~~`_internal` directory required, will see if I can package this into a single exe on the next release~~ (fixed in v1.3.1) -- ~~Custom command line parameters~~ (added in v1.3.0) -- ~~More iMatrix generation parameters~~ (added in v1.3.0) -- ~~Specify multiple KV overrides~~ (added in v1.1.0) -- ~~Better error handling~~ (added in v1.1.0) -- ~~Cannot select output/token embd type~~ (fixed in v1.1.0) -- ~~Importing presets with KV overrides causes UI thread crash~~ (fixed in v1.3.0) - -## Planned features: + +## Planned Features + - Actual progress bar tracking -- Download safetensors from HF and convert to unquanted GGUF +- Download safetensors from HF and convert to unquantized GGUF - Perplexity testing - Managing shards (coming in the next release) -- Time estimated for quantization -- Dynamic values for KV cache, e.g. autogguf.quantized.time=str:{system.time.milliseconds} (coming in the next release) -- Ability to select and start multiple quants at once (saved in presets) (coming in the next release) +- Time estimation for quantization +- Dynamic values for KV cache (coming in the next release) +- Ability to select and start multiple quants at once (saved in presets, coming in the next release) + +## Troubleshooting -## Troubleshooting: -- ~~llama.cpp quantizations errors out with an iostream error: create the `quantized_models` directory (or set a directory)~~ (fixed in v1.2.1, automatically created on launch) -- SSL module cannot be found error: Install OpenSSL or run from source `python src/main.py` using the `run.bat` script (`pip install requests`) +- SSL module cannot be found error: Install OpenSSL or run from source using `python src/main.py` with the `run.bat` script (`pip install requests`) -## Contributing: -Simply fork the repo and make your changes; when merging make sure to have the latest commits. Description should contain a changelog of what's new. +## Contributing + +Fork the repo, make your changes, and ensure you have the latest commits when merging. Include a changelog of new features in your pull request description. + +## User Interface -## User interface: ![image](https://github.com/user-attachments/assets/2660c841-07ba-4c3f-ae3a-e63c7068bdc1) -## Stargazers: +## Stargazers + [![Star History Chart](https://api.star-history.com/svg?repos=leafspark/AutoGGUF&type=Date)](https://star-history.com/#leafspark/AutoGGUF&Date)