The Perplexity AI Toolkit makes it easy to use Perplexity Labs' Sonar language models (built on top of Meta's latest and most advanced model LLama-3.1) for creating chatbots, generating text, and searching the web (in real-time). It's designed for everyone, from beginners to experienced developers, allowing quick addition of AI features to projects with simple commands. While it offers simplicity and lightweight integration, it doesn't compromise on power; experienced developers can access the full suite of advanced options available via the API, ensuring robust customization and control. This toolkit is perfect for those looking to efficiently tap into advanced AI without getting bogged down in technical details, yet it still provides the depth needed for complex project requirements.
- Conversational AI: Create interactive, real-time chat experiences (chatbots) or AI assistants.
- Real-Time Web Search: Conduct online searches in real-time with precise query responses.
- Highly Customizable: Tailor settings like streaming output, system prompts, sampling temperature and more to suit your specific requirements.
- Lightweight Integration: Efficiently designed with minimal dependencies, requiring only the
requestspackage for core functionality.
Python 3.x- An API key from Perplexity AI
The following Python packages are required:
requests: For making HTTP requests to the Perplexity API.
The following Python packages are optional:
python-dotenv: For managing API keys and other environment variables.
To use the Perplexity AI Toolkit, clone the repository to your local machine and install the required Python packages.
Clone the repository:
git clone https://github.com/RMNCLDYO/perplexity-ai-toolkit.gitNavigate to the repositories folder:
cd perplexity-ai-toolkitInstall the required dependencies:
pip install -r requirements.txt-
Obtain an API key from Perplexity.
-
You have three options for managing your API key:
Click here to view the API key configuration options
-
Setting it as an environment variable on your device (recommended for everyday use)
- Navigate to your terminal.
- Add your API key like so:
export PERPLEXITY_API_KEY=your_api_key
This method allows the API key to be loaded automatically when using the wrapper or CLI.
-
Using an .env file (recommended for development):
- Install python-dotenv if you haven't already:
pip install python-dotenv. - Create a .env file in the project's root directory.
- Add your API key to the .env file like so:
PERPLEXITY_API_KEY=your_api_key
This method allows the API key to be loaded automatically when using the wrapper or CLI, assuming you have python-dotenv installed and set up correctly.
- Install python-dotenv if you haven't already:
-
Direct Input:
-
If you prefer not to use a
.envfile, you can directly pass your API key as an argument to the CLI or the wrapper functions.CLI
--api_key "your_api_key"Wrapper
api_key="your_api_key"
This method requires manually inputting your API key each time you initiate an API call, ensuring flexibility for different deployment environments.
-
-
The Perplexity AI Toolkit can be used in two different modes: Chat, and Search. Each mode is designed for specific types of interactions with the language models.
Chat mode is intended for chatting with an AI model (similar to a chatbot) or building conversational applications.
CLI
python cli.py --chatWrapper
from perplexity import Chat
Chat().run()An executable version of this example can be found here. (You must move this file to the root folder before running the program.)
Search mode is intended for searching online (in real-time) for a single query as perplexity does not support multi-turn conversations with their online models.
CLI
python cli.py --search --query "What is today's date?"Wrapper
from perplexity import Search
Search().run(query="What is today's date?")An executable version of this example can be found here. (You must move this file to the root folder before running the program.)
Search mode is limited to 'online' models, such as llama-3.1-sonar-small-128k-online, llama-3.1-sonar-large-128k-online and llama-3.1-sonar-huge-128k-online.
| Description | CLI Flags | CLI Usage | Wrapper Usage |
|---|---|---|---|
| Enable chat mode | -c, --chat |
--chat | See mode usage above |
| Enable online search mode | -s, --search |
--search | See mode usage above |
| Online search query | -q, --query |
--query "What is today's date?" | query="What is today's date?" |
| User prompt | -p, --prompt |
--prompt "How many stars are there in our galaxy?" | prompt="How many stars are there in our galaxy?" |
| API key for authentication | -a, --api_key |
--api_key your_api_key | api_key="your_api_key" |
| Model name | -m, --model |
--model "llama-3.1-sonar-small-128k-chat" | model="llama-3.1-sonar-small-128k-chat" |
| Enable streaming mode | -st, --stream |
--stream | stream=True |
| System prompt (instructions) | -sp, --system_prompt |
--system_prompt "Be precise and concise." | system_prompt="Be precise and concise." |
| Maximum tokens to generate | -mt, --max_tokens |
--max_tokens 100 | max_tokens=100 |
| Sampling temperature | -tm, --temperature |
--temperature 0.7 | temperature=0.7 |
| Nucleus sampling threshold | -tp, --top_p |
--top_p 0.9 | top_p=0.9 |
| Top-k sampling threshold | -tk, --top_k |
--top_k 40 | top_k=40 |
| Penalize tokens based on their presence | -pp, --presence_penalty |
--presence_penalty 0.5 | presence_penalty=0.5 |
| Penalize tokens based on their frequency | -fp, --frequency_penalty |
--frequency_penalty 0.5 | frequency_penalty=0.5 |
To exit the program at any time, you can type
exitorquit. This command works similarly whether you're interacting with the program via the CLI or through the Python wrapper ensuring that you can easily and safely conclude your work with the Perplexity AI Toolkit without having to resort to interrupt signals or forcibly closing the terminal or command prompt.
Perplexity offers both native models and a selection of large, open-source instruct models.
| Model | Parameter Count | Context Length |
|---|---|---|
llama-3.1-sonar-small-128k-online |
8B | 127,072 |
llama-3.1-sonar-large-128k-online |
70B | 127,072 |
llama-3.1-sonar-huge-128k-online |
405B | 127,072 |
- Perplexity makes note that the search subsystem of the Online LLMs do not attend to the system prompt. You can only use the system prompt to provide instructions related to style, tone, and language of the response.
| Model | Parameter Count | Context Length |
|---|---|---|
llama-3.1-sonar-small-128k-chat |
8B | 131,072 |
llama-3.1-sonar-large-128k-chat |
70B | 131,072 |
| Model | Parameter Count | Context Length |
|---|---|---|
llama-3.1-8b-instruct |
8B | 131,072 |
llama-3.1-70b-instruct |
70B | 131,072 |
- Where possible, Perplexity tries to match the Hugging Face implementation.
Contributions are welcome!
Please refer to CONTRIBUTING.md for detailed guidelines on how to contribute to this project.
Encountered a bug? We'd love to hear about it. Please follow these steps to report any issues:
- Check if the issue has already been reported.
- Use the Bug Report template to create a detailed report.
- Submit the report here.
Your report will help us make the project better for everyone.
Got an idea for a new feature? Feel free to suggest it. Here's how:
- Check if the feature has already been suggested or implemented.
- Use the Feature Request template to create a detailed request.
- Submit the request here.
Your suggestions for improvements are always welcome.
Stay up-to-date with the latest changes and improvements in each version:
- CHANGELOG.md provides detailed descriptions of each release.
Your security is important to us. If you discover a security vulnerability, please follow our responsible disclosure guidelines found in SECURITY.md. Please refrain from disclosing any vulnerabilities publicly until said vulnerability has been reported and addressed.
Licensed under the MIT License. See LICENSE for details.