SambaNova Integration¶
Instructor supports SambaNova's LLM API, allowing you to use structured outputs with their models.
Installation¶
Basic Usage¶
from openai import OpenAI
import instructor
import os
from pydantic import BaseModel
client = instructor.from_openai(
OpenAI(
base_url="https://api.sambanova.ai/v1",
api_key=os.environ["SAMBANOVA_API_KEY"]
)
)
class User(BaseModel):
name: str
age: int
user = client.chat.completions.create(
model="Meta-Llama-3.1-405B-Instruct",
messages=[
{"role": "user", "content": "Ivan is 28"},
],
response_model=User,
)
print(user)
# > User(name='Ivan', age=28)
Async Usage¶
from openai import AsyncOpenAI
import instructor
import os
from pydantic import BaseModel
client = instructor.from_openai(
AsyncOpenAI(
base_url="https://api.sambanova.ai/v1",
api_key=os.environ["SAMBANOVA_API_KEY"]
)
)
class User(BaseModel):
name: str
age: int
async def get_user():
user = await client.chat.completions.create(
model="Meta-Llama-3.1-405B-Instruct",
messages=[
{"role": "user", "content": "Ivan is 28"},
],
response_model=User,
)
return user
# Run with asyncio
import asyncio
user = asyncio.run(get_user())
print(user)
# > User(name='Ivan', age=28)
Available Models¶
Check the SambaNova documentation for the latest model offerings and capabilities.