SOLA is a Smart Offline-first LLM Assistant.
By adopting an offline-first approach to data and model execution, privacy is guaranteed.
Still, its integration with LLM AI models and access to external (online) tools make it smart.
The authors of the project are:
More information on the project can be found in the report and slides included in the repository.
Follow the Setup Guide to install all requirements.
The voice assistant lives in the main script.
It can be started using a CLI interface.
Usage: main.py [OPTIONS]
Local-First LLM Voice Assistant
╭─ Options ───────────────────────────────────────────────────╮
│ --config-path TEXT [default: ./config.json] │
│ --log-level TEXT [default: INFO] │
│ --help Show this message and exit. │
╰─────────────────────────────────────────────────────────────╯The voice assistant can be configured by editing the config.json file.
Check out the Configuration Guide for more details.
A number of test scripts are available to verify the working of the individual components in the pipeline.
Run a test by calling the test script from the main directory.
python tests/test_[something].pyThe performance of a tool-llm on various sample tasks can be evaluated using the evaluate_toolllm.py script.
The results from our experiments can be found in the logs directory.
The pipeline that powers the voice assistant stands on the shoulders of giants:
- CLI Tool built on Typer
- Wakeword Detection with openWakeWord
- Automatic Speech Recognition / Speech-to-Text with:
- OpenAI Whisper
- Vosk
- Coqui STT (Formerly Mozilla DeepSpeech)
- Large Language Model (LLM) integration with:
- LlamaEdge
- Use Llama 3 8B for the best experience.
- A notebook for fine-tuning TinyLlama for tool usage is included in the
finetuningdirectory. The fine-tuned weights can be downloaded from Google Drive.
- OpenAI GPT 3/4
- LlamaEdge
- Speech Synthesis / Text-to-Speech with:
- Piper
- Coqui TTS (Formerly Mozilla TTS)
- MycroftAI/mimic3
This project is supposed to be a proof-of-concept.
There are a lot of interesting projects out there that are developed to a more mature point.
Go check them out!
