Provider Integrations¶

Instructor works with many different LLM providers, allowing you to use structured outputs with your preferred models.

Major Cloud Providers

Leading AI providers with comprehensive features

OpenAI · Azure · Anthropic · Google.GenerativeAI · Vertex AI · AWS Bedrock · Google.GenAI
Additional Cloud Providers

Other commercial AI providers with specialized offerings

Cohere · Mistral · DeepSeek · Together AI · Groq · Fireworks · Cerebras · Writer · Perplexity Sambanova
Open Source

Run open-source models locally or in the cloud

Ollama · llama-cpp-python
Routing

Unified interfaces for multiple providers

LiteLLM OpenRouter

Common Features¶

All integrations support these core features:

Feature	Description	Documentation
Model Patching	Enhance provider clients with structured output capabilities	Patching
Response Models	Define expected response schema with Pydantic	Models
Validation	Ensure responses match your schema definition	Validation
Streaming	Stream partial or iterative responses	Partial, Iterable
Hooks	Add callbacks for monitoring and debugging	Hooks

However, each provider has different capabilities and limitations. Refer to the specific provider documentation for details.

Providers support different methods for generating structured outputs:

Mode	Description	Providers
`TOOLS`	Uses OpenAI-style tools/function calling	OpenAI, Anthropic, Mistral
`PARALLEL_TOOLS`	Multiple simultaneous tool calls	OpenAI
`JSON`	Direct JSON response generation	OpenAI, Gemini, Cohere, Perplexity
`MD_JSON`	JSON embedded in markdown	Most providers
`BEDROCK_TOOLS`	AWS Bedrock function calling	AWS Bedrock
`BEDROCK_JSON`	AWS Bedrock JSON generation	AWS Bedrock
`PERPLEXITY_JSON`	Perplexity JSON generation	Perplexity

See the Modes Comparison guide for details.

To use a provider with Instructor:

Install the required dependencies:

pip install "instructor[provider]"  # e.g., instructor[anthropic]

Import the provider client and patch it with Instructor:

import instructor
from provider_package import Client

client = instructor.from_provider(Client())

Use the patched client with your Pydantic model:

response = client.chat.completions.create(
    response_model=YourModel,
    messages=[{"role": "user", "content": "Your prompt"}]
)

For provider-specific setup and examples, visit each provider's documentation page.

If you need assistance with a specific integration: