Hooks¶

Hooks provide a powerful mechanism for intercepting and handling events during the completion and parsing process in the Instructor library. They allow you to add custom behavior, logging, or error handling at various stages of the API interaction.

Overview¶

The Hooks system in Instructor is based on the Hooks class, which manages event registration and emission. It supports several predefined events that correspond to different stages of the completion and parsing process.

Supported Hook Events¶

`completion:kwargs`¶

This hook is emitted when completion arguments are provided. It receives all arguments passed to the completion function. These will contain the model, messages, tools, AFTER any response_model or validation_context parameters have been converted to their respective values.

def handler(*args, **kwargs) -> None: ...

`completion:response`¶

This hook is emitted when a completion response is received. It receives the raw response object from the completion API.

def handler(response) -> None: ...

`completion:error`¶

This hook is emitted when an error occurs during completion before any retries are attempted and the response is parsed as a pydantic model.

def handler(error) -> None: ...

`parse:error`¶

This hook is emitted when an error occurs during parsing of the response as a pydantic model. This can happen if the response is not valid or if the pydantic model is not compatible with the response.

def handler(error) -> None: ...

`completion:last_attempt`¶

This hook is emitted when the last retry attempt is made.

def handler(error) -> None: ...

Implementation Details¶

The Hooks system is implemented in the instructor/hooks.py file. The Hooks class handles the registration and emission of hook events. You can refer to this file to see how hooks work under the hood. The retry logic that uses Hooks is implemented in the instructor/retry.py file. This shows how Hooks are used when trying again after errors during completions.

Registering Hooks¶

You can register hooks using the on method of the Instructor client or a Hooks instance. Here's an example:

import instructor
import openai
import pprint

client = instructor.from_openai(openai.OpenAI())


def log_completion_kwargs(*args, **kwargs):
    pprint.pprint({"args": args, "kwargs": kwargs})


client.on("completion:kwargs", log_completion_kwargs)

resp = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello, world!"}],
    response_model=str,
)
print(resp)
#> Hello, world!

Emitting Events¶

Events are automatically emitted by the Instructor library at appropriate times. You don't need to manually emit events in most cases.

Removing Hooks¶

You can remove a specific hook using the off method:

client.off("completion:kwargs", log_completion_kwargs)

Clearing Hooks¶

To remove all hooks for a specific event or all events:

# Clear hooks for a specific event
client.clear("completion:kwargs")

# Clear all hooks
client.clear()

Example: Logging and Debugging¶

Here's a comprehensive example demonstrating how to use hooks for logging and debugging:

import instructor
import openai
import pydantic


def log_completion_kwargs(kwargs) -> None:
    print("## Completion kwargs:")
    print(kwargs)
    """
    {
        "messages": [
            {
                "role": "user",
                "content": "Extract the user name and age from the following text: 'John is 20 years old'",
            }
        ],
        "model": "gpt-4o-mini",
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "User",
                    "description": "Correctly extracted `User` with all the required parameters with correct types",
                    "parameters": {
                        "properties": {
                            "name": {"title": "Name", "type": "string"},
                            "age": {"title": "Age", "type": "integer"},
                        },
                        "required": ["age", "name"],
                        "type": "object",
                    },
                },
            }
        ],
        "tool_choice": {"type": "function", "function": {"name": "User"}},
    }
    """


def log_completion_response(response) -> None:
    print("## Completion response:")
    #> ## Completion response:
    """
    {
        'id': 'chatcmpl-AWl4Mj5Jrv7m7JkOTIiHXSldQIOFm',
        'choices': [
            {
                'finish_reason': 'stop',
                'index': 0,
                'logprobs': None,
                'message': {
                    'content': None,
                    'refusal': None,
                    'role': 'assistant',
                    'audio': None,
                    'function_call': None,
                    'tool_calls': [
                        {
                            'id': 'call_6oQ9WXxeSiVEV71B9IYtsbIE',
                            'function': {
                                'arguments': '{"name":"John","age":-1}',
                                'name': 'User',
                            },
                            'type': 'function',
                        }
                    ],
                },
            }
        ],
        'created': 1732370794,
        'model': 'gpt-4o-mini-2024-07-18',
        'object': 'chat.completion',
        'service_tier': None,
        'system_fingerprint': 'fp_0705bf87c0',
        'usage': {
            'completion_tokens': 10,
            'prompt_tokens': 87,
            'total_tokens': 97,
            'completion_tokens_details': {
                'audio_tokens': 0,
                'reasoning_tokens': 0,
                'accepted_prediction_tokens': 0,
                'rejected_prediction_tokens': 0,
            },
            'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0},
        },
    }
    """
    print(response.model_dump())
    """
    {
        'id': 'chatcmpl-AWl4Mxdq0BUGRlVCA61z8YOIVga7F',
        'choices': [
            {
                'finish_reason': 'stop',
                'index': 0,
                'logprobs': None,
                'message': {
                    'content': None,
                    'refusal': None,
                    'role': 'assistant',
                    'audio': None,
                    'function_call': None,
                    'tool_calls': [
                        {
                            'id': 'call_EJIEr27Mb6sdbplnYw4iBWlm',
                            'function': {
                                'arguments': '{"name":"John","age":10}',
                                'name': 'User',
                            },
                            'type': 'function',
                        }
                    ],
                },
            }
        ],
        'created': 1732370794,
        'model': 'gpt-4o-mini-2024-07-18',
        'object': 'chat.completion',
        'service_tier': None,
        'system_fingerprint': 'fp_0705bf87c0',
        'usage': {
            'completion_tokens': 9,
            'prompt_tokens': 87,
            'total_tokens': 96,
            'completion_tokens_details': {
                'audio_tokens': 0,
                'reasoning_tokens': 0,
                'accepted_prediction_tokens': 0,
                'rejected_prediction_tokens': 0,
            },
            'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0},
        },
    }
    """


def log_completion_error(error) -> None:
    print("## Completion error:")
    print({"error": error})


def log_parse_error(error) -> None:
    print("## Parse error:")
    #> ## Parse error:
    print(error)
    """
    1 validation error for User
    age
    Value error, Age cannot be negative [type=value_error, input_value=-10, input_type=int]
        For further information visit https://errors.pydantic.dev/2.8/v/value_error
    """


# Create an Instructor client
client = instructor.from_openai(openai.OpenAI())

client.on("completion:kwargs", log_completion_kwargs)
client.on("completion:response", log_completion_response)

client.on("completion:error", log_completion_error)
client.on("parse:error", log_parse_error)


# Define a model with a validator
class User(pydantic.BaseModel):
    name: str
    age: int

    @pydantic.field_validator("age")
    def check_age(cls, v: int) -> int:
        if v < 0:
            raise ValueError("Age cannot be negative")
        return v


try:
    # Use the client to create a completion
    user = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
    #> ## Parse error:
                "role": "user",
                "content": "Extract the user name and age from the following text: 'John is -1 years old'",
    """
    1 validation error for User
    age
      Value error, Age cannot be negative [type=value_error, input_value=-1, input_type=int]
        For further information visit https://errors.pydantic.dev/2.9/v/value_error
    """
            }
        ],
        response_model=User,
        max_retries=1,
    )
except Exception as e:
    print(f"Error: {e}")


user = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {
            "role": "user",
            "content": "Extract the user name and age from the following text: 'John is 10 years old'",
        }
    ],
    response_model=User,
    max_retries=1,
)
print(user)
#> name='John' age=10
    """
    Error: 1 validation error for User
    age
      Value error, Age cannot be negative [type=value_error, input_value=-1, input_type=int]
        For further information visit https://errors.pydantic.dev/2.9/v/value_error
    """

This example demonstrates:

Defining hook handlers for different events.
Registering the hooks with the Instructor client.
Using a Pydantic model with a validator.
Making a completion request that will trigger various hooks.

The hooks will log information at different stages of the process, helping with debugging and understanding the flow of data.

Best Practices¶

Error Handling: Always include error handling in your hook handlers to prevent exceptions from breaking the main execution flow. We will automatically warn if an exception is raised in a hook handler.
Performance: Keep hook handlers lightweight to avoid impacting the performance of the main application.
Modularity: Use hooks to separate concerns. For example, use hooks for logging, monitoring, or custom business logic without cluttering the main code.
Consistency: Use the same naming conventions and patterns across all your hooks for better maintainability.
Documentation: Document the purpose and expected input/output of each hook handler for easier collaboration and maintenance.

By leveraging hooks effectively, you can create more flexible, debuggable, and maintainable applications when working with the Instructor library and language models.