Prompete

Prompete is a wrapper for LiteLLM. It integrates LLMEasyTools, and Jinja2 templates to create a flexible system for managing prompts and chat interactions. Part of the API is inspired by Claudette.

The key idea behind Prompete is that LLM prompts contain two distinct components:

• data: the information that the LLM is supposed to manipulate
• instructions: the instructions on how to interpret and use that data

These two parts have different nature, and are manipulated differently. They change in different rythms and in different phases of the project. They are edited in different ways and mixing them together is error prone and makes the prompt management harder. For example long text blocks in code demolish the visual clues a programmer relies on when reading that code.

Prompete tries to address this by separating the two components into data structures and templates.

But that does not mean that when using Prompete you need to start immediately with templated prompts. You can start working in Prompete with simple string prompts, then progressively adopt more advanced features as your needs evolve.

Features

• Template-based prompt generation using Jinja2
• Integration with various LLM APIs through LiteLLM
• Emulate response_format by using tools (for models that don't support response_format)
• Easy function calling with LLMEasyTools
• Conversation management with the Chat interface
• System prompts and custom prompt roles

Installation

Install Prompete using pip:

pip install prompete

Quick Start

First you need to set up your API credentials in your environment variables. Depending on the model you choose, you'll need to set:

• OPENAI_API_KEY for OpenAI models
• ANTHROPIC_API_KEY for Anthropic models

For example:

export OPENAI_API_KEY='your-openai-api-key-here'

Basic example

text prompt

from prompete import Chat

model = "gpt-4o-mini"  # OpenAI model

# Create a Chat instance with a system prompt
chat = Chat(
    model=model,
    system_prompt="You are a helpful assistant specializing in Python programming."
)

# Start the conversation
user_message = "What's the difference between a list and a tuple in Python?"
response = chat(user_message)

print("User:", user_message)
print("AI:", response)

Templating example

A full example of templating with Prompete can be found in the examples/templating.py file.

To use templating with Prompete, you need to create a Jinja2 Environment. You can use all features of Jinja2 like multiple directories to search for template files, add custom filters, etc.

# Create a Jinja2 Environment instance with multiple template directories
renderer = Environment(
    loader=ChoiceLoader([
        FileSystemLoader(os.path.join(current_dir, "templates")),
    ])
)

The data is passed to the template as a subclass of Prompt.

@dataclass(frozen=True)
class TaskPrompt(Prompt):
    user_name: str
    language: str
    task: str

task_prompt = TaskPrompt(
    user_name="Alice",
    language="Python",
    task="write a function to calculate the factorial of a number"
)

print(chat(task_prompt))

The prompt tempalte is found by looking up the class name in the templates defined in the renderer. The template can use the prompt fields as variables.

Function Calling

Prompete integrates with LLMEasyTools to provide function calling capabilities. Here's how it works:

Basic Function Calling

from prompete import Chat

def get_weather(location: str, unit: str = "celsius") -> dict:
    """Get the current weather in a given location"""
    # Simulate weather API call
    return {
        "location": location,
        "temperature": 22,
        "unit": unit,
        "forecast": ["sunny", "windy"],
    }

chat = Chat(model="gpt-4")
response = chat("Should I bring an umbrella to London today?", tools=[get_weather])
print(tool_results)

Two Ways to Handle Tool Results

When working with function calls, there are two main approaches to handle the results:

1. LLM Interpretation: Let the LLM interpret the tool results and provide a human-friendly response

# LLM will call the function and interpret results
chat = Chat(model="gpt-4")
response = chat("Should I bring an umbrella to London today?", tools=[get_weather])
print(response)  # LLM provides a natural language response based on the weather data

The pitfall of this more 'agentic' approach is that the LLM can sometimes decide to guess the weather instead of calling the function.

2. Direct Code Processing: Handle the tool results directly in your code

# Get direct access to both LLM response and tool results
chat = Chat(model="gpt-4")
chat.append("What's the weather in London?")
response, tool_results = chat.complete_once(tools=[get_weather])
# response is empty when using OpenAI models - but might be non-empty when using Anthropic models
if tool_results:
    weather_data = tool_results[0]
    print(f"Temperature: {weather_data['temperature']}°{weather_data['unit']}")

Control Over Tool Execution

There are two ways to control tool execution loops:

• max_llm_requests: - parameter to __call__ - limits how many LLM request can be made in a single conversation turn
  • default is 2 - first request to generate a tool call message - second one to interpret the tool call results if the LLM does not return a tool call message, the second request will be skipped you can set it to more than 2 if you want the LLM to try tool execution untill it gets the results it needs
• tool_choice: - parameter to __call__ and compelete_once passed to LiteLLM - you can use it to force the LLM to use a specific tool

Tool Results in Conversation History

All tool calls and their results are automatically saved in the chat history, making them available for context in future conversation turns.

Key Concepts

• Chat: The main class for managing conversations and interacting with LLMs.
• Prompt: Base class for creating custom prompt types.
• renderer: Jinja2 Environment for rendering prompts with dynamic content.

Contributing

We welcome contributions to Prompete! Please follow these steps:

1. Fork the repository
2. Create a new branch for your feature or bug fix
3. Make your changes and write tests if applicable
4. Submit a pull request with a clear description of your changes

For major changes, please open an issue first to discuss the proposed changes.

Writing Test Cases

We strongly encourage writing test cases for both bug reports and feature requests:

• For bugs: Include a test case that reproduces the issue, showing expected vs. actual behavior.
• For features: Provide test cases describing the desired functionality, including inputs and expected outputs.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Support

If you encounter any issues or have questions, please open an issue on our GitHub repository.

Acknowledgements

Prompete is built upon several excellent libraries:

• LiteLLM for universal LLM API support
• Jinja2 for powerful templating capabilities
• Claudette for the API inspiration

We're grateful to the maintainers and contributors of these projects.