Structured Output for Open Source and Local LLMS
Originally, Instructor facilitated API interactions solely via the OpenAI SDK, with an emphasis on function call by incorporating Pydantic for structured data validation and serialization.
As the year progressed, we expanded our toolkit by integrating JSON mode, thus enhancing our adaptability to vision models and open source models. This advancement now enables us to support an extensive range of models, from GPT and Mistral to virtually any model accessible through Ollama and Hugging Face, facilitated by llama-cpp-python. For more insights into leveraging JSON mode with various models, refer back to our detailed guide on Patching.
If you want to check out a course on how to use Instructor with Pydantic, check out our course on Steering language models towards structured outputs..