Instructor Now Supports llms.txt
We've added automatic llms.txt
generation to Instructor's documentation using the mkdocs-llmstxt
plugin.
🎉 Introducing Kura: Turn your chat logs into actionable insights! Discover user patterns, extract intents, and understand conversation flows at scale. Try it on GitHub →
If you want to get updates on new features and tips on how to use Instructor, you can subscribe to our newsletter below to get notified when we publish new content.
We've added automatic llms.txt
generation to Instructor's documentation using the mkdocs-llmstxt
plugin.
Today we integrated the mkdocs-llmstxt
plugin into Instructor's documentation pipeline. This powerful plugin automatically generates llms.txt
files from our MkDocs documentation, making our comprehensive guides instantly accessible to AI language models.
Semantic validation uses LLMs to evaluate content against complex, subjective, and contextual criteria that would be difficult to implement with traditional rule-based validation approaches.
As LLMs become increasingly integrated into production systems, ensuring the quality and safety of their outputs is paramount. Traditional validation methods relying on explicit rules can't keep up with the complexity and nuance of natural language. With the release of Instructor's semantic validation capabilities, we now have a powerful way to validate structured outputs against sophisticated criteria.
We're excited to announce Instructor's integration with OpenAI's new Responses API. This integration brings a more streamlined approach to working with structured outputs from OpenAI models. Let's see what makes this integration special and how it can improve your LLM applications.
We are pleased to introduce a significant enhancement to Instructor: the from_provider()
function. While Instructor has always focused on providing robust structured outputs, we've observed that many users work with multiple LLM providers. This often involves repetitive setup for each client.
The from_provider()
function aims to simplify this process, making it easier to initialize clients and experiment across different models.
This new feature offers a streamlined, string-based method to initialize an Instructor-enhanced client for a variety of popular LLM providers.
Anthropic's new web search tool, when combined with Instructor, provides a powerful way to get real-time, structured data from the web. This allows you to build applications that can answer questions and provide information that is up-to-date, going beyond the knowledge cut-off of large language models.
In this post, we'll explore how to use the web_search
tool with Instructor to fetch the latest information and structure it into a Pydantic model. Even a simple structure can be very effective for clarity and further processing.
With OpenAI joining Anthropic in supporting the Model Context Protocol (MCP), we're witnessing a unified standard for language models to interact with external systems. This creates exciting opportunities for multi-LLM architectures where specialized AI applications work in parallel-discovering tools, handing off tasks, and accessing powerful capabilities through standardized interfaces.
We're excited to announce that Instructor now implements the llms.txt specification! You can now find our llms.txt file at python.useinstructor.com/llms.txt. This adoption marks an important step in making our documentation more accessible to AI language models.
AI-assisted coding is changing how we use version control. Many developers now use what I call "vibe coding" - coding with AI help. This creates new challenges with Git. Today I'll share how we're using Cursor rules in Instructor to solve these problems.
New in v1.9.1: Instructor now ships with built-in caching support for all providers. Simply pass a cache adapter when creating your client to dramatically reduce API costs and improve response times.
Starting with Instructor v1.9.1, we've introduced native caching support that makes optimization effortless. Instead of implementing complex caching decorators or wrapper functions, you can now pass a cache adapter directly to from_provider()
and automatically cache all your structured LLM calls.
Before v1.9.1, caching required custom decorators and manual implementation. Now, it's as simple as:
from instructor import from_provider
from instructor.cache import AutoCache
# Works with any provider - caching flows through automatically
client = from_provider(
"openai/gpt-4o",
cache=AutoCache(maxsize=1000)
)
# Your normal calls are now cached automatically
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
first = client.create(
messages=[{"role": "user", "content": "Extract: John is 25"}],
response_model=User
)
second = client.create(
messages=[{"role": "user", "content": "Extract: John is 25"}],
response_model=User
)
# second call was served from cache - same result, zero cost!
assert first.name == second.name
The beauty of native caching is that it works with every provider through the same simple API:
from instructor.cache import AutoCache, DiskCache
# Works with OpenAI
openai_client = from_provider("openai/gpt-3.5-turbo", cache=AutoCache())
# Works with Anthropic
anthropic_client = from_provider("anthropic/claude-3-haiku", cache=AutoCache())
# Works with Google
google_client = from_provider("google/gemini-pro", cache=DiskCache())
# Works with any provider in the ecosystem
groq_client = from_provider("groq/llama-3.1-8b", cache=AutoCache())
No provider-specific configuration needed. The cache parameter flows through **kwargs
to all underlying implementations automatically.
Instructor v1.9.1 ships with two production-ready cache implementations:
Perfect for single-process applications and development:
from instructor.cache import AutoCache
# Thread-safe in-memory cache with LRU eviction
cache = AutoCache(maxsize=1000)
client = from_provider("openai/gpt-4o", cache=cache)
When to use: - Development and testing - Single-process applications - When you need maximum speed (200,000x+ faster cache hits) - Applications where cache persistence isn't required
Ideal when you need cache persistence across sessions:
from instructor.cache import DiskCache
# Persistent disk-based cache
cache = DiskCache(directory=".instructor_cache")
client = from_provider("anthropic/claude-3-sonnet", cache=cache)
When to use: - Applications that restart frequently - Development workflows where you want to preserve cache between sessions - When working with expensive or time-intensive API calls - Local applications with moderate performance requirements
Instructor automatically generates intelligent cache keys that include:
This means when you update your Pydantic model (adding fields, changing descriptions, etc.), the cache automatically invalidates old entries - no stale data!
from instructor.cache import make_cache_key
# Generate deterministic cache key
key = make_cache_key(
messages=[{"role": "user", "content": "hello"}],
model="gpt-3.5-turbo",
response_model=User,
mode="TOOLS"
)
print(key) # SHA-256 hash: 9b8f5e2c8c9e...
Want Redis, Memcached, or a custom backend? Simply inherit from BaseCache
:
from instructor.cache import BaseCache
import redis
class RedisCache(BaseCache):
def __init__(self, host="localhost", port=6379, **kwargs):
self.redis = redis.Redis(host=host, port=port, **kwargs)
def get(self, key: str):
value = self.redis.get(key)
return value.decode() if value else None
def set(self, key: str, value, ttl: int | None = None):
if ttl:
self.redis.setex(key, ttl, value)
else:
self.redis.set(key, value)
# Use your custom cache
redis_cache = RedisCache(host="my-redis-server")
client = from_provider("openai/gpt-4o", cache=redis_cache)
The BaseCache
interface is intentionally minimal - just implement get()
and set()
methods and you're ready to go.
Control cache expiration with per-call TTL overrides:
# Cache this result for 1 hour
result = client.create(
messages=[{"role": "user", "content": "Generate daily report"}],
response_model=Report,
cache_ttl=3600 # 1 hour in seconds
)
TTL support depends on your cache backend: - AutoCache: TTL is ignored (no expiration) - DiskCache: Full TTL support with automatic expiration - Custom backends: Implement TTL handling in your set()
method
If you were using custom caching decorators, migrating is straightforward:
Before v1.9.1:
@functools.cache
def extract_user(text: str) -> User:
return client.create(
messages=[{"role": "user", "content": text}],
response_model=User
)
With v1.9.1:
# Remove decorator, add cache to client
client = from_provider("openai/gpt-4o", cache=AutoCache())
def extract_user(text: str) -> User:
return client.create(
messages=[{"role": "user", "content": text}],
response_model=User
)
No more function-level caching logic - just create your client with caching enabled and all calls benefit automatically.
Native caching delivers the same dramatic performance improvements you'd expect:
For a comprehensive deep-dive into caching strategies and performance analysis, check out our complete caching guide.
Ready to enable native caching? Here's your quick start:
Upgrade to v1.9.1+:
Choose your cache backend:
Add cache to your client:
Use normally - caching happens automatically:
For detailed information about cache design, custom implementations, and advanced patterns, visit our Caching Concepts documentation.
The native caching feature represents our commitment to making high-performance LLM applications simple and accessible. No more complex caching logic - just fast, cost-effective structured outputs out of the box.
Have questions about native caching or want to share your use case? Join the discussion in our GitHub repository or check out the complete documentation.