Skip to content

Structured outputs with Fireworks, a complete guide w/ instructor

Fireworks provides efficient and cost-effective AI models with enterprise-grade reliability. This guide shows you how to use Instructor with Fireworks's models for type-safe, validated responses.

Quick Start

Install Instructor with Fireworks support:

pip install "instructor[fireworks-ai]"

Simple User Example (Sync)

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel

# Initialize the client
client = Fireworks()

# Enable instructor patches
client = instructor.from_fireworks(client)


class User(BaseModel):
    name: str
    age: int


# Create structured output
user = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Extract: Jason is 25 years old",
        }
    ],
    model="accounts/fireworks/models/llama-v3-8b-instruct",
    response_model=User,
)

print(user)
# > User(name='Jason', age=25)

Simple User Example (Async)

from fireworks.client import AsyncFireworks
import instructor
from pydantic import BaseModel
import asyncio

# Initialize async client
client = AsyncFireworks()

# Enable instructor patches
client = instructor.from_fireworks(client)


class User(BaseModel):
    name: str
    age: int


async def extract_user():
    user = await client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "Extract: Jason is 25 years old",
            }
        ],
        model="accounts/fireworks/models/llama-v3-8b-instruct",
        response_model=User,
    )
    return user


# Run async function
user = asyncio.run(extract_user())
print(user)  # User(name='Jason', age=25)

Nested Example

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel


# Enable instructor patches
client = instructor.from_fireworks(Fireworks())


class Address(BaseModel):
    street: str
    city: str
    country: str


class User(BaseModel):
    name: str
    age: int
    addresses: list[Address]


# Create structured output with nested objects
user = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": """
                Extract: Jason is 25 years old.
                He lives at 123 Main St, New York, USA
                and has a summer house at 456 Beach Rd, Miami, USA
            """,
        }
    ],
    model="accounts/fireworks/models/llama-v3-8b-instruct",
    response_model=User,
)

print(user)
#> {
#>     'name': 'Jason',
#>     'age': 25,
#>     'addresses': [
#>         {
#>             'street': '123 Main St',
#>             'city': 'New York',
#>             'country': 'USA'
#>         },
#>         {
#>             'street': '456 Beach Rd',
#>             'city': 'Miami',
#>             'country': 'USA'
#>         }
#>     ]
#> }

Streaming Support

Instructor has two main ways that you can use to stream responses out

  1. Iterables: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
  2. Partial Streaming: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.

Partial Streaming Example

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel


# Enable instructor patches
client = instructor.from_fireworks(Fireworks())


class User(BaseModel):
    name: str
    age: int
    bio: str


user = client.chat.completions.create_partial(
    model="accounts/fireworks/models/llama-v3-8b-instruct",
    messages=[
        {
            "role": "user",
            "content": "Create a user profile for Jason + 1 sentence bio, age 25",
        },
    ],
    response_model=User,
)

for user_partial in user:
    print(user_partial)
    # name=None age=None bio=None
    # name='Jason' age=None bio=None
    # name='Jason' age=25 bio="When he's"
    # name='Jason' age=25 bio="When he's not working as a graphic designer, Jason can usually be found trying out new craft beers or attempting to cook something other than ramen noodles."

Iterable Example

from fireworks.client import Fireworks
import instructor
from pydantic import BaseModel


# Enable instructor patches
client = instructor.from_fireworks(Fireworks())


class User(BaseModel):
    name: str
    age: int


# Extract multiple users from text
users = client.chat.completions.create_iterable(
    model="accounts/fireworks/models/llama-v3-8b-instruct",
    messages=[
        {
            "role": "user",
            "content": """
            Extract users:
            1. Jason is 25 years old
            2. Sarah is 30 years old
            3. Mike is 28 years old
        """,
        },
    ],
    response_model=User,
)

for user in users:
    print(user)

    # name='Jason' age=25
    # name='Sarah' age=30
    # name='Mike' age=28

Instructor Modes

We provide several modes to make it easy to work with the different response models that Fireworks supports

  1. instructor.Mode.FIREWORKS_JSON : This parses the raw text completion into a pydantic object
  2. instructor.Mode.FIREWORKS_TOOLS : This uses Fireworks's tool calling API to return structured outputs to the client

Updates and Compatibility

Instructor maintains compatibility with Fireworks's latest API versions. Check the changelog for updates.

Note: Always verify model-specific features and limitations before implementing streaming functionality in production environments.