Data Model

The TensorZero Gateway stores inference and feedback data in ClickHouse. This data can be used for observability, experimentation, and optimization.

ChatInference

The ChatInference table stores information about inference requests for Chat Functions made to the TensorZero Gateway.

A ChatInference row can be associated with one or more ModelInference rows, depending on the variant’s type. For chat_completion, there will be a one-to-one relationship between rows in the two tables. For other variant types, there might be more associated rows.

Column	Type	Notes
id	UUID	Must be a UUIDv7
function_name	String
variant_name	String
episode_id	UUID	Must be a UUIDv7
input	String (JSON)	input field in the /inference request body
output	String (JSON)	Array of content blocks
tool_params	String (JSON)	Object with any tool parameters (e.g. tool_choice, available_tools) used for the inference
inference_params	String (JSON)	Object with any inference parameters per variant type (e.g. {"chat_completion": {"temperature": 0.5}})
processing_time_ms	UInt32
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

JsonInference

The JsonInference table stores information about inference requests for JSON Functions made to the TensorZero Gateway.

A JsonInference row can be associated with one or more ModelInference rows, depending on the variant’s type. For chat_completion, there will be a one-to-one relationship between rows in the two tables. For other variant types, there might be more associated rows.

Column	Type	Notes
id	UUID	Must be a UUIDv7
function_name	String
variant_name	String
episode_id	UUID	Must be a UUIDv7
input	String (JSON)	input field in the /inference request body
output	String (JSON)	Object with parsed and raw fields
output_schema	String (JSON)	Schema that the output must conform to
inference_params	String (JSON)	Object with any inference parameters per variant type (e.g. {"chat_completion": {"temperature": 0.5}})
processing_time_ms	UInt32
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

ModelInference

The ModelInference table stores information about each inference request to a model provider. This is the inference request you’d make if you had called the model provider directly.

Column	Type	Notes
id	UUID	Must be a UUIDv7
inference_id	UUID	Must be a UUIDv7
raw_request	String	Raw request as sent to the model provider (varies)
raw_response	String	Raw response from the model provider (varies)
model_name	String	Name of the model used for the inference
model_provider_name	String	Name of the model provider used for the inference
input_tokens	UInt32
output_tokens	UInt32
response_time_ms	UInt32
ttft_ms	Nullable(UInt32)	Only available in streaming inferences
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

InferenceById

The InferenceById table is a materialized view that combines data from ChatInference and JSONInference. Notably, it indexes the table by id for fast lookup.

Column	Type	Notes
id	UUID	Must be a UUIDv7
function_name	String
variant_name	String
episode_id	UUID	Must be a UUIDv7
type	String	Either ‘chat’ or ‘json’

DynamicInContextLearningExample

The DynamicInContextLearningExample table stores examples for dynamic in-context learning variants.

Column	Type	Notes
id	UUID	Must be a UUIDv7
function_name	LowCardinality(String)
variant_name	LowCardinality(String)
namespace	String
input	String (JSON)
output	String
embedding	Array(Float32)
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

BooleanMetricFeedback

The BooleanMetricFeedback table stores feedback for metrics of type = "boolean".

Column	Type	Notes
id	UUID	Must be a UUIDv7
target_id	UUID	Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config
metric_name	String
value	Bool
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

FloatMetricFeedback

The FloatMetricFeedback table stores feedback for metrics of type = "float".

Column	Type	Notes
id	UUID	Must be a UUIDv7
target_id	UUID	Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config
metric_name	String
value	Float32
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

CommentFeedback

The CommentFeedback table stores feedback provided with metric_name of "comment". Comments are free-form text feedbacks.

Column	Type	Notes
id	UUID	Must be a UUIDv7
target_id	UUID	Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config
target_type	"inference" or "episode"
value	String
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)

DemonstrationFeedback

The DemonstrationFeedback table stores feedback in the form of demonstrations. Demonstrations are examples of good behaviors.

Column	Type	Notes
id	UUID	Must be a UUIDv7
inference_id	UUID	Must be a UUIDv7
value	String	The demonstration or example provided as feedback (must match function output)
timestamp	DateTime	Materialized from id (using UUIDv7ToDateTime function)