Skip to content

Performance Optimization

Introducing structured outputs with Cerebras Inference

What's Cerebras?

Cerebras offers the fastest inference on the market, 20x faster than on GPUs.

Sign up for a Cerebras Inference API key here at cloud.cerebras.ai.

Basic Usage

To get guaranteed structured outputs with Cerebras Inference, you

  1. Create a new Instructor client with the from_cerebras method
  2. Define a Pydantic model to pass into the response_model parameter
  3. Get back a validated response exactly as you would expect

You'll also need to install the Cerebras SDK to use the client. You can install it with the command below.

Introduction to Caching in Python

Instructor makes working with language models easy, but they are still computationally expensive.

Today, we're diving into optimizing instructor code while maintaining the excellent DX offered by Pydantic models. We'll tackle the challenges of caching Pydantic models, typically incompatible with pickle, and explore solutions that use decorators like functools.cache. Then, we'll craft custom decorators with diskcache and redis to support persistent caching and distributed systems.