Cerebras offers the fastest inference on the market, 20x faster than on GPUs.
Sign up for a Cerebras Inference API key here at cloud.cerebras.ai.
To get guaranteed structured outputs with Cerebras Inference, you
Instructor makes working with language models easy, but they are still computationally expensive.
Today, we're diving into optimizing instructor code while maintaining the excellent DX offered by Pydantic models. We'll tackle the challenges of caching Pydantic models, typically incompatible with pickle, and explore solutions that use decorators like functools.cache. Then, we'll craft custom decorators with diskcache and redis to support persistent caching and distributed systems.