Frequently Asked Questions¶

This page answers common questions about using Instructor with various LLM providers.

General Questions¶

What is Instructor?¶

Instructor is a library that makes it easy to get structured data from Large Language Models (LLMs). It uses Pydantic to define output schemas and provides a consistent interface across different LLM providers.

How does Instructor work?¶

Instructor "patches" LLM clients to add a response_model parameter that accepts a Pydantic model. When you make a request, Instructor:

Converts your Pydantic model to a schema the LLM can understand
Formats the prompt appropriately for the provider
Validates the LLM's response against your model
Retries automatically if validation fails
Returns a properly typed Pydantic object

Which LLM providers does Instructor support?¶

Instructor supports many providers, including:

OpenAI (GPT models)
Anthropic (Claude models)
Google (Gemini models)
Cohere
Mistral AI
Groq
LiteLLM (meta-provider)
Various open-source models via Ollama, llama.cpp, etc.

See the Integrations section for the complete list.

What's the difference between various modes?¶

Instructor supports different modes for different providers:

Mode.TOOLS - Uses the OpenAI function calling API (recommended for OpenAI)
Mode.JSON - Instructs the model to return JSON directly
Mode.ANTHROPIC_TOOLS - Uses Anthropic's tool calling feature
Mode.GEMINI_TOOLS - Uses Gemini's function calling

The optimal mode depends on your provider and use case. See Patching for details.

Installation and Setup¶

How do I install Instructor?¶

Basic installation:

pip install instructor

For specific providers:

pip install "instructor[anthropic]"  # For Anthropic
pip install "instructor[google-generativeai]"  # For Google/Gemini

What environment variables do I need?¶

This depends on your provider:

OpenAI: OPENAI_API_KEY
Anthropic: ANTHROPIC_API_KEY
Google: GOOGLE_API_KEY

Each provider has specific requirements documented in their integration guide.

Common Issues¶

Why is my model not returning structured data?¶

Common reasons include:

Using the wrong mode for your provider
Complex schema that confuses the model
Insufficient context in your prompt
Using a model that doesn't support function/tool calling

Try simplifying your schema or providing clearer instructions in your prompt.

How do I handle validation errors?¶

Instructor automatically retries when validation fails. You can customize this behavior:

from tenacity import stop_after_attempt

result = client.chat.completions.create(
    response_model=MyModel,
    max_retries=stop_after_attempt(5),  # Retry up to 5 times
    messages=[...]
)

Can I see the raw response from the LLM?¶

Yes, use create_with_completion:

result, completion = client.chat.completions.create_with_completion(
    response_model=MyModel,
    messages=[...]
)

result is your Pydantic model, and completion is the raw response.

How do I stream large responses?¶

Use create_partial for partial updates as the response is generated:

stream = client.chat.completions.create_partial(
    response_model=MyModel,
    messages=[...]
)

for partial in stream:
    print(partial)  # Partial model with fields filled in as they're generated

Performance and Costs¶

How can I optimize token usage?¶

Use concise prompts
Use smaller models for simpler tasks
Use the MD_JSON or JSON mode for simple schemas
Cache responses for repeated queries

How do I handle rate limits?¶

Instructor uses the tenacity library for retries, which you can configure:

from tenacity import retry_if_exception_type, wait_exponential
from openai.error import RateLimitError

result = client.chat.completions.create(
    response_model=MyModel,
    max_retries=retry_if_exception_type(RateLimitError),
    messages=[...],
)

Advanced Usage¶

How do I use Instructor with FastAPI?¶

Instructor works seamlessly with FastAPI:

from fastapi import FastAPI
from pydantic import BaseModel
import instructor
from openai import OpenAI

app = FastAPI()
client = instructor.from_openai(OpenAI())

class UserInfo(BaseModel):
    name: str
    age: int

@app.post("/extract")
async def extract_user_info(text: str) -> UserInfo:
    return client.chat.completions.create(
        model="gpt-3.5-turbo",
        response_model=UserInfo,
        messages=[{"role": "user", "content": text}]
    )

How do I use Instructor with async code?¶

Use the async client:

import instructor
import asyncio
from openai import AsyncOpenAI

client = instructor.from_openai(AsyncOpenAI())

async def extract_data():
    result = await client.chat.completions.create(
        response_model=MyModel,
        messages=[...]
    )
    return result

asyncio.run(extract_data())