TensorZero Autopilot is an automated AI engineer that analyzes LLM observability data, optimizes prompts and models, sets up evals, and runs A/B tests. Join the waitlist →
Supervised Fine-Tuning (SFT) trains a language model on curated examples of good behavior, resulting in a custom model that performs better on your specific use case.
TensorZero simplifies the SFT workflow by helping you curate training data from your historical inferences and feedback, then launching fine-tuning jobs on your preferred provider.Here’s how it works:
You collect examples of good LLM behavior (demonstrations or inferences with good metrics).
TensorZero renders these examples using your prompt templates into a training dataset.
TensorZero uploads the dataset and launches a fine-tuning job on your provider (OpenAI, GCP Vertex AI, Fireworks, or Together).
The provider trains a custom model and returns a model identifier.
You update your configuration to use the fine-tuned model.
Supervised fine-tuning is particularly useful when you have substantial high-quality data and want to improve model behavior beyond what prompting alone can achieve.
Criterion
Impact
Details
Complexity
Low
Requires data curation; few parameters
Data Efficiency
Moderate
Requires hundreds to thousands of high-quality examples
Optimization Ceiling
High
Can significantly improve model behavior beyond prompting
Optimization Cost
Moderate
More expensive than DICL, but relatively cost effective
Inference Cost
Low
Fine-tuned models typically cost the same as the base model
Inference Latency
Low
No runtime overhead
SFT tends to work best when:
You have hundreds to thousands of high-quality examples.
Example: Data Extraction (Named Entity Recognition) — Configuration
system_template.minijinja
Copy
You are an assistant that is performing a named entity recognition task.Your job is to extract entities from a given text.The entities you are extracting are:- people- organizations- locations- miscellaneous other entitiesPlease return the entities in the following JSON format:{"person": ["person1", "person2", ...],"organization": ["organization1", "organization2", ...],"location": ["location1", "location2", ...],"miscellaneous": ["miscellaneous1", "miscellaneous2", ...]}
2
Collect your optimization data
TensorZero Dataset
Historical Inferences
After deploying the TensorZero Gateway with ClickHouse, build a dataset of good examples for the extract_entities function you configured.
You can create datapoints from historical inferences or external/synthetic datasets.
After deploying the TensorZero Gateway with ClickHouse, make inference calls to the extract_entities function you configured.
TensorZero automatically collects structured data about those inferences, which can later be used as training examples for SFT.You can curate good examples in multiple ways:
Collecting demonstrations: Collect demonstrations of good behavior (or labels) from human annotation or other sources.
Filtering with metrics: Query inferences that scored well on your metrics (e.g. output_source="inference" with a filter for high scores).
Examples from an expensive model: Run inferences with a powerful model (e.g. GPT-4o) and use those outputs as demonstrations for a smaller model (e.g. GPT-4o Mini).
SFT performance depends heavily on data quality.
There is a trade-off between dataset size and quality of datapoints.
For this example, we’ll use demonstrations.
You can submit demonstration feedback using the demonstration metric:
Copy
t0.feedback( metric_name="demonstration", value=corrected_output, # Provide the ideal output for that inference inference_id=response.inference_id,)
Then, query inferences with output_source="demonstration" to get examples where the output has been corrected:
from tensorzero import TogetherSFTConfigoptimization_config = TogetherSFTConfig( model="deepseek-ai/DeepSeek-R1-Distill-Qwen-14B", n_epochs=3, # optional)
Together uses credentials from the TOGETHER_API_KEY environment variable by default.
Optional Weights & Biases integration can be configured in tensorzero.toml:
Fine-tuning typically takes 10-30 minutes for small datasets, but can take hours for large datasets.
You can close your script and poll later using the job handle.
7
Update your configuration with the fine-tuned model
After optimization completes, extract the fine-tuned model name and update your configuration:
