Hooks¶
Hooks provide a powerful mechanism for intercepting and handling events during the completion and parsing process in the Instructor library. They allow you to add custom behavior, logging, or error handling at various stages of the API interaction.
Overview¶
The Hooks system in Instructor is based on the Hooks class, which manages event registration and emission. It supports several predefined events that correspond to different stages of the completion and parsing process.
Supported Hook Events¶
completion:kwargs¶
This hook is emitted when completion arguments are provided. It receives all arguments passed to the completion function. These will contain the model, messages, tools, AFTER any response_model or validation_context parameters have been converted to their respective values.
completion:response¶
This hook is emitted when a completion response is received. It receives the raw response object from the completion API.
completion:error¶
This hook is emitted when an error occurs during completion before any retries are attempted and the response is parsed as a pydantic model.
parse:error¶
This hook is emitted when an error occurs during parsing of the response as a pydantic model. This can happen if the response is not valid or if the pydantic model is not compatible with the response.
completion:last_attempt¶
This hook is emitted when the last retry attempt is made.
Implementation Details¶
The Hooks system is implemented in the instructor/hooks.py file. The Hooks class handles the registration and emission of hook events. You can refer to this file to see how hooks work under the hood. The retry logic that uses Hooks is implemented in the instructor/retry.py file. This shows how Hooks are used when trying again after errors during completions.
Registering Hooks¶
You can register hooks using the on method of the Instructor client or a Hooks instance. Here's an example:
import instructor
import openai
import pprint
client = instructor.from_openai(openai.OpenAI())
def log_completion_kwargs(*args, **kwargs):
pprint.pprint({"args": args, "kwargs": kwargs})
client.on("completion:kwargs", log_completion_kwargs)
resp = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello, world!"}],
response_model=str,
)
print(resp)
#> Hello! How can I assist you today?
Emitting Events¶
Events are automatically emitted by the Instructor library at appropriate times. You don't need to manually emit events in most cases.
Removing Hooks¶
You can remove a specific hook using the off method:
Clearing Hooks¶
To remove all hooks for a specific event or all events:
# Clear hooks for a specific event
client.clear("completion:kwargs")
# Clear all hooks
client.clear()
Example: Logging and Debugging¶
Here's a comprehensive example demonstrating how to use hooks for logging and debugging:
import instructor
import openai
import pydantic
def log_completion_kwargs(*args, **kwargs) -> None:
"""Log the completion kwargs."""
print("## Completion kwargs:")
#> ## Completion kwargs:
"""
{
'args': (),
'kwargs': {
'messages': [
{
'role': 'user',
'content': "Extract the user name and age from the following text: 'John is -1 years old'",
}
],
'model': 'gpt-3.5-turbo',
'tools': [
{
'type': 'function',
'function': {
'name': 'User',
'description': 'Correctly extracted `User` with all the required parameters with correct types',
'parameters': {
'properties': {
'name': {'title': 'Name', 'type': 'string'},
'age': {'title': 'Age', 'type': 'integer'},
},
'required': ['age', 'name'],
'type': 'object',
},
},
}
],
'tool_choice': {'type': 'function', 'function': {'name': 'User'}},
},
}
"""
print({"args": args, "kwargs": kwargs})
"""
{
'args': (),
'kwargs': {
'messages': [
{
'role': 'user',
'content': "Extract the user name and age from the following text: 'John is -1 years old'",
},
{
'role': 'assistant',
'content': '',
'tool_calls': [
{
'id': 'call_GamMbnNfWFCHPHGgZy2VxD09',
'function': {
'arguments': '{"name":"John","age":-1}',
'name': 'User',
},
'type': 'function',
}
],
},
{
'role': 'tool',
'tool_call_id': 'call_GamMbnNfWFCHPHGgZy2VxD09',
'name': 'User',
'content': 'Validation Error found:\n1 validation error for User\nage\n Value error, Age cannot be negative [type=value_error, input_value=-1, input_type=int]\n For further information visit https://errors.pydantic.dev/2.8/v/value_error\nRecall the function correctly, fix the errors',
},
],
'model': 'gpt-3.5-turbo',
'tools': [
{
'type': 'function',
'function': {
'name': 'User',
'description': 'Correctly extracted `User` with all the required parameters with correct types',
'parameters': {
'properties': {
'name': {'title': 'Name', 'type': 'string'},
'age': {'title': 'Age', 'type': 'integer'},
'error_message': {
'anyOf': [{'type': 'string'}, {'type': 'null'}],
'default': None,
'title': 'Error Message',
},
},
'required': ['age', 'name'],
'type': 'object',
},
},
}
],
'tool_choice': {'type': 'function', 'function': {'name': 'User'}},
},
}
"""
#> ## Completion response:
def log_completion_response(response) -> None:
"""
{
'id': 'chatcmpl-AH1sl7JTZ37B8RrriiFBjH1n3mNVL',
'choices': [
{
'finish_reason': 'stop',
'index': 0,
'logprobs': None,
'message': {
'content': None,
'refusal': None,
'role': 'assistant',
'function_call': None,
'tool_calls': [
{
'id': 'call_GamMbnNfWFCHPHGgZy2VxD09',
'function': {
'arguments': '{"name":"John","age":-1}',
'name': 'User',
},
'type': 'function',
}
],
},
}
],
'created': 1728622175,
'model': 'gpt-3.5-turbo-0125',
'object': 'chat.completion',
'service_tier': None,
'system_fingerprint': None,
'usage': {
'completion_tokens': 9,
'prompt_tokens': 106,
'total_tokens': 115,
'completion_tokens_details': {'audio_tokens': None, 'reasoning_tokens': 0},
'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0},
},
}
"""
print("## Completion response:")
#> ## Completion response:
print(response.model_dump())
"""
{
'id': 'chatcmpl-AH1smoYP5X5dR7JM83lU78BQemcy1',
'choices': [
{
'finish_reason': 'stop',
'index': 0,
'logprobs': None,
'message': {
'content': None,
'refusal': None,
'role': 'assistant',
'function_call': None,
'tool_calls': [
{
'id': 'call_VlkoWrL5PMcBd4JIR6cdqxu6',
'function': {
'arguments': '{"name":"John","age":1}',
'name': 'User',
},
'type': 'function',
}
],
},
}
],
'created': 1728622176,
'model': 'gpt-3.5-turbo-0125',
'object': 'chat.completion',
'service_tier': None,
'system_fingerprint': None,
'usage': {
'completion_tokens': 9,
'prompt_tokens': 193,
'total_tokens': 202,
'completion_tokens_details': {'audio_tokens': None, 'reasoning_tokens': 0},
'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0},
},
}
"""
def log_completion_error(error) -> None:
"""Log the completion error."""
print("## Completion error:")
print({"error": error})
#> ## Parse error:
"""
{'error': 1 validation error for User
age
Value error, Age cannot be negative [type=value_error, input_value=-1, input_type=int]
For further information visit https://errors.pydantic.dev/2.8/v/value_error}
"""
def log_parse_error(error) -> None:
"""Log the parse error."""
print("## Parse error:")
print({"error": error})
# Create an Instructor client
client = instructor.from_openai(openai.OpenAI())
# Register hooks
client.on("completion:kwargs", log_completion_kwargs)
client.on("completion:response", log_completion_response)
client.on("completion:error", log_completion_error)
client.on("parse:error", log_parse_error)
# Define a model with a validator
class User(pydantic.BaseModel):
name: str
age: int
@pydantic.field_validator("age")
def check_age(cls, v: int) -> int:
if v < 0:
raise ValueError("Age cannot be negative")
return v
# Use the client to create a completion
user = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{
"role": "user",
"content": "Extract the user name and age from the following text: 'John is -1 years old'",
}
],
response_model=User,
)
print(user)
#> name='John' age=1
This example demonstrates:
- Defining hook handlers for different events.
- Registering the hooks with the Instructor client.
- Using a Pydantic model with a validator.
- Making a completion request that will trigger various hooks.
The hooks will log information at different stages of the process, helping with debugging and understanding the flow of data.
Best Practices¶
-
Error Handling: Always include error handling in your hook handlers to prevent exceptions from breaking the main execution flow. We will automatically warn if an exception is raised in a hook handler.
-
Performance: Keep hook handlers lightweight to avoid impacting the performance of the main application.
-
Modularity: Use hooks to separate concerns. For example, use hooks for logging, monitoring, or custom business logic without cluttering the main code.
-
Consistency: Use the same naming conventions and patterns across all your hooks for better maintainability.
-
Documentation: Document the purpose and expected input/output of each hook handler for easier collaboration and maintenance.
By leveraging hooks effectively, you can create more flexible, debuggable, and maintainable applications when working with the Instructor library and language models.