Using the Command Line Interface¶
The instructor CLI provides functionalities for managing fine-tuning jobs on OpenAI.
Incomplete API
The CLI is still under development and does not yet support all features of the API. If you would like to use a feature that is not yet supported, please consider using the contributing to our library jxnl/instructor instead.
!!! note "Low hanging fruit"
If you want to contribute we're looking for a few things:
1. Adding filenames on upload
Creating a Fine-Tuning Job¶
View Jobs Options¶
$ instructor jobs --help
Usage: instructor jobs [OPTIONS] COMMAND [ARGS]...
Monitor and create fine tuning jobs
โญโ Options โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ --help Display the help message. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Commands โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ cancel Cancel a fine-tuning job. โ
โ create-from-file Create a fine-tuning job from a file. โ
โ create-from-id Create a fine-tuning job from an existing ID. โ
โ list Monitor the status of the most recent fine-tuning jobs. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
Create from File¶
The create-from-file command uploads and trains a model in a single step.
โฏ instructor jobs create-from-file --help
Usage: instructor jobs create-from-file [OPTIONS] FILE
Create a fine-tuning job from a file.
โญโ Arguments โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ * file TEXT Path to the file for fine-tuning [default: None] [required] โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Options โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ --model TEXT Model to use for fine-tuning [default: gpt-3.5-turbo] โ
โ --poll INTEGER Polling interval in seconds [default: 2] โ
โ --n-epochs INTEGER Number of epochs for fine-tuning โ
โ --batch-size TEXT Batch size for fine-tuning โ
โ --learning-rate-multiplier TEXT Learning rate multiplier for fine-tuning โ
โ --validation-file TEXT Path to the validation file [default: None] โ
โ --model-suffix TEXT Suffix to identify the model [default: None] โ
โ --help Show this message and exit. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Usage¶
$ instructor jobs create-from-file transformed_data.jsonl --validation_file validation_data.jsonl --n_epochs 3 --batch_size 16 --learning_rate_multiplier 0.5
Create from ID¶
The create-from-id command uses an uploaded file and trains a model
โฏ instructor jobs create-from-id --help
Usage: instructor jobs create-from-id [OPTIONS] ID
Create a fine-tuning job from an existing ID.
โญโ Arguments โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ * id TEXT ID of the existing fine-tuning job [default: None] [required] โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Options โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ --model TEXT Model to use for fine-tuning โ
โ [default: gpt-3.5-turbo] โ
โ --n-epochs INTEGER Number of epochs for fine-tuning โ
โ --batch-size TEXT Batch size for fine-tuning โ
โ --learning-rate-multiplier TEXT Learning rate multiplier for fine-tuning โ
โ --validation-file-id TEXT ID of the uploaded validation file โ
โ [default: None] โ
โ --help Show this message and exit. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
Usage¶
$ instructor files upload transformed_data.jsonl
$ instructor files upload validation_data.jsonl
$ instructor files list
...
$ instructor jobs create_from_id <file_id> --validation_file <validation_file_id> --n_epochs 3 --batch_size 16 --learning_rate_multiplier 0.5
Viewing Files and Jobs¶
Viewing Jobs¶
$ instructor jobs list
OpenAI Fine Tuning Job Monitoring
โโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโณโโโโโโโโโณโโโโโโโโโโโโโโโโโโ
โ โ โ โ Completion โ โ โ โ โ
โ Job ID โ Status โ Creation Time โ Time โ Model Name โ File ID โ Epochs โ Base Model โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ ftjob-PWo6uwkโฆ โ ๐ซ cancelled โ 2023-08-23 โ N/A โ โ file-F7lJg6Z4โฆ โ 3 โ gpt-3.5-turbo-โฆ โ
โ โ โ 23:10:54 โ โ โ โ โ โ
โ ftjob-1whjva8โฆ โ ๐ซ cancelled โ 2023-08-23 โ N/A โ โ file-F7lJg6Z4โฆ โ 3 โ gpt-3.5-turbo-โฆ โ
โ โ โ 22:47:05 โ โ โ โ โ โ
โ ftjob-wGoBDldโฆ โ ๐ซ cancelled โ 2023-08-23 โ N/A โ โ file-F7lJg6Z4โฆ โ 3 โ gpt-3.5-turbo-โฆ โ
โ โ โ 22:44:12 โ โ โ โ โ โ
โ ftjob-yd5aRTcโฆ โ โ
succeeded โ 2023-08-23 โ 2023-08-23 โ ft:gpt-3.5-turโฆ โ file-IQxAUDqXโฆ โ 3 โ gpt-3.5-turbo-โฆ โ
โ โ โ 14:26:03 โ 15:02:29 โ โ โ โ โ
โโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโดโโโโโโโโโดโโโโโโโโโโโโโโโโโโ
Automatically refreshes every 5 seconds, press Ctrl+C to exit
Viewing Files¶
$ instructor files list
OpenAI Files
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโณโโโโโโโโโโโโ
โ File ID โ Size (bytes) โ Creation Time โ Filename โ Purpose โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ file-0lw2BSNRUlXZXRRu2beCCWjl โ 369523 โ 2023-08-23 23:31:57 โ file โ fine-tune โ
โ file-IHaUXcMEykmFUp1kt2puCDEq โ 369523 โ 2023-08-23 23:09:35 โ file โ fine-tune โ
โ file-ja9vRBf0FydEOTolaa3BMqES โ 369523 โ 2023-08-23 22:42:29 โ file โ fine-tune โ
โ file-F7lJg6Z47CREvmx4kyvyZ6Sn โ 369523 โ 2023-08-23 22:42:03 โ file โ fine-tune โ
โ file-YUxqZPyJRl5GJCUTw3cNmA46 โ 369523 โ 2023-08-23 22:29:10 โ file โ fine-tune โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโดโโโโโโโโโโโโ
Contributions¶
We aim to provide a light wrapper around the API rather than offering a complete CLI. Contributions are welcome! Please feel free to make an issue at jxnl/instructor/issues or submit a pull request.