Introducing Hooks

[jxnl]

Hooks

Hooks provide a powerful mechanism for intercepting and handling events during the completion and parsing process in the Instructor library. They allow you to add custom behavior, logging, or error handling at various stages of the API interaction.

Overview

The Hooks system in Instructor is based on the Hooks class, which manages event registration and emission. It supports several predefined events that correspond to different stages of the completion and parsing process.

Supported Hook Events

completion:kwargs

This hook is emitted when completion arguments are provided. It receives all arguments passed to the completion function. These will contain the model, messages, tools, AFTER any response_model or validation_context parameters have been converted to their respective values.

def handler(*args, **kwargs) -> None: ...

completion:response

This hook is emitted when a completion response is received. It receives the raw response object from the completion API.

def handler(response) -> None: ...

completion:error

This hook is emitted when an error occurs during completion before any retries are attempted and the response is parsed as a pydantic model.

def handler(error) -> None: ...

parse:error

This hook is emitted when an error occurs during parsing of the response as a pydantic model. This can happen if the response is not valid or if the pydantic model is not compatible with the response.

def handler(error) -> None: ...

completion:last_attempt

This hook is emitted when the last retry attempt is made.

def handler(error) -> None: ...

Registering Hooks

You can register hooks using the on method of the Instructor client or a Hooks instance. Here's an example:

import instructor
import openai
import pprint

client = instructor.from_openai(openai.OpenAI())


def log_completion_kwargs(*args, **kwargs):
    pprint.pprint({"args": args, "kwargs": kwargs})


client.on("completion:kwargs", log_completion_kwargs)

resp = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello, world!"}],
    response_model=str,
)
print(resp)
#> Hello! How can I assist you today?

Emitting Events

Events are automatically emitted by the Instructor library at appropriate times. You don't need to manually emit events in most cases.

Removing Hooks

You can remove a specific hook using the off method:

client.off("completion:kwargs", log_completion_kwargs)

Clearing Hooks

To remove all hooks for a specific event or all events:

# Clear hooks for a specific event
client.clear("completion:kwargs")

# Clear all hooks
client.clear()

Example: Logging and Debugging

Here's a comprehensive example demonstrating how to use hooks for logging and debugging:

import instructor
import openai
import pydantic


def log_completion_kwargs(*args, **kwargs) -> None:
    """Log the completion kwargs."""
    print("## Completion kwargs:")
    #> ## Completion kwargs:
    """
    {
        'args': (),
        'kwargs': {
            'messages': [
                {
                    'role': 'user',
                    'content': "Extract the user name and age from the following text: 'John is -1 years old'",
                }
            ],
            'model': 'gpt-3.5-turbo',
            'tools': [
                {
                    'type': 'function',
                    'function': {
                        'name': 'User',
                        'description': 'Correctly extracted `User` with all the required parameters with correct types',
                        'parameters': {
                            'properties': {
                                'name': {'title': 'Name', 'type': 'string'},
                                'age': {'title': 'Age', 'type': 'integer'},
                            },
                            'required': ['age', 'name'],
                            'type': 'object',
                        },
                    },
                }
            ],
            'tool_choice': {'type': 'function', 'function': {'name': 'User'}},
        },
    }
    """
    print({"args": args, "kwargs": kwargs})
    """
    {
        'args': (),
        'kwargs': {
            'messages': [
                {
                    'role': 'user',
                    'content': "Extract the user name and age from the following text: 'John is -1 years old'",
                },
                {
                    'role': 'assistant',
                    'content': '',
                    'tool_calls': [
                        {
                            'id': 'call_GamMbnNfWFCHPHGgZy2VxD09',
                            'function': {
                                'arguments': '{"name":"John","age":-1}',
                                'name': 'User',
                            },
                            'type': 'function',
                        }
                    ],
                },
                {
                    'role': 'tool',
                    'tool_call_id': 'call_GamMbnNfWFCHPHGgZy2VxD09',
                    'name': 'User',
                    'content': 'Validation Error found:\n1 validation error for User\nage\n  Value error, Age cannot be negative [type=value_error, input_value=-1, input_type=int]\n    For further information visit https://errors.pydantic.dev/2.8/v/value_error\nRecall the function correctly, fix the errors',
                },
            ],
            'model': 'gpt-3.5-turbo',
            'tools': [
                {
                    'type': 'function',
                    'function': {
                        'name': 'User',
                        'description': 'Correctly extracted `User` with all the required parameters with correct types',
                        'parameters': {
                            'properties': {
                                'name': {'title': 'Name', 'type': 'string'},
                                'age': {'title': 'Age', 'type': 'integer'},
                                'error_message': {
                                    'anyOf': [{'type': 'string'}, {'type': 'null'}],
                                    'default': None,
                                    'title': 'Error Message',
                                },
                            },
                            'required': ['age', 'name'],
                            'type': 'object',
                        },
                    },
                }
            ],
            'tool_choice': {'type': 'function', 'function': {'name': 'User'}},
        },
    }
    """


    #> ## Completion response:
def log_completion_response(response) -> None:
    """
    {
        'id': 'chatcmpl-AH1sl7JTZ37B8RrriiFBjH1n3mNVL',
        'choices': [
            {
                'finish_reason': 'stop',
                'index': 0,
                'logprobs': None,
                'message': {
                    'content': None,
                    'refusal': None,
                    'role': 'assistant',
                    'function_call': None,
                    'tool_calls': [
                        {
                            'id': 'call_GamMbnNfWFCHPHGgZy2VxD09',
                            'function': {
                                'arguments': '{"name":"John","age":-1}',
                                'name': 'User',
                            },
                            'type': 'function',
                        }
                    ],
                },
            }
        ],
        'created': 1728622175,
        'model': 'gpt-3.5-turbo-0125',
        'object': 'chat.completion',
        'service_tier': None,
        'system_fingerprint': None,
        'usage': {
            'completion_tokens': 9,
            'prompt_tokens': 106,
            'total_tokens': 115,
            'completion_tokens_details': {'audio_tokens': None, 'reasoning_tokens': 0},
            'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0},
        },
    }
    """
    print("## Completion response:")
    #> ## Completion response:
    print(response.model_dump())
    """
    {
        'id': 'chatcmpl-AH1smoYP5X5dR7JM83lU78BQemcy1',
        'choices': [
            {
                'finish_reason': 'stop',
                'index': 0,
                'logprobs': None,
                'message': {
                    'content': None,
                    'refusal': None,
                    'role': 'assistant',
                    'function_call': None,
                    'tool_calls': [
                        {
                            'id': 'call_VlkoWrL5PMcBd4JIR6cdqxu6',
                            'function': {
                                'arguments': '{"name":"John","age":1}',
                                'name': 'User',
                            },
                            'type': 'function',
                        }
                    ],
                },
            }
        ],
        'created': 1728622176,
        'model': 'gpt-3.5-turbo-0125',
        'object': 'chat.completion',
        'service_tier': None,
        'system_fingerprint': None,
        'usage': {
            'completion_tokens': 9,
            'prompt_tokens': 193,
            'total_tokens': 202,
            'completion_tokens_details': {'audio_tokens': None, 'reasoning_tokens': 0},
            'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0},
        },
    }
    """


def log_completion_error(error) -> None:
    """Log the completion error."""
    print("## Completion error:")
    print({"error": error})

    #> ## Parse error:

    """
    {'error': 1 validation error for User
    age
      Value error, Age cannot be negative [type=value_error, input_value=-1, input_type=int]
        For further information visit https://errors.pydantic.dev/2.8/v/value_error}
    """
def log_parse_error(error) -> None:
    """Log the parse error."""
    print("## Parse error:")
    print({"error": error})


# Create an Instructor client
client = instructor.from_openai(openai.OpenAI())

# Register hooks
client.on("completion:kwargs", log_completion_kwargs)
client.on("completion:response", log_completion_response)
client.on("completion:error", log_completion_error)
client.on("parse:error", log_parse_error)


# Define a model with a validator
class User(pydantic.BaseModel):
    name: str
    age: int

    @pydantic.field_validator("age")
    def check_age(cls, v: int) -> int:
        if v < 0:
            raise ValueError("Age cannot be negative")
        return v


# Use the client to create a completion
user = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "user",
            "content": "Extract the user name and age from the following text: 'John is -1 years old'",
        }
    ],
    response_model=User,
)

print(user)
#> name='John' age=1

This example demonstrates:

1. Defining hook handlers for different events.
2. Registering the hooks with the Instructor client.
3. Using a Pydantic model with a validator.
4. Making a completion request that will trigger various hooks.

The hooks will log information at different stages of the process, helping with debugging and understanding the flow of data.

Best Practices

1. Error Handling: Always include error handling in your hook handlers to prevent exceptions from breaking the main execution flow. We will automatically warn if an exception is raised in a hook handler.

2. Performance: Keep hook handlers lightweight to avoid impacting the performance of the main application.

3. Modularity: Use hooks to separate concerns. For example, use hooks for logging, monitoring, or custom business logic without cluttering the main code.

4. Consistency: Use the same naming conventions and patterns across all your hooks for better maintainability.

5. Documentation: Document the purpose and expected input/output of each hook handler for easier collaboration and maintenance.

By leveraging hooks effectively, you can create more flexible, debuggable, and maintainable applications when working with the Instructor library and language models.

[S4mpl3r]

Does a hook handler function run concurrently with the main thread or things are handled sequentially?

[S4mpl3r]

@jxnl Would appreciate an insight into this

[jxnl]

Currently sequentially it's in the hooks that pie class. I didn't want to get too complicated for the first implementation.