# TensorZero Docs

## Docs

- [Comparison: TensorZero vs. DSPy](https://www.tensorzero.com/docs/comparison/dspy.md): TensorZero is an open-source alternative to DSPy featuring an LLM gateway, observability, optimization, evaluations, and experimentation.
- [Comparison: TensorZero vs. LangChain](https://www.tensorzero.com/docs/comparison/langchain.md): TensorZero is an open-source alternative to LangChain featuring an LLM gateway, observability, optimization, evaluations, and experimentation.
- [Comparison: TensorZero vs. Langfuse](https://www.tensorzero.com/docs/comparison/langfuse.md): TensorZero is an open-source alternative to Langfuse featuring an LLM gateway, observability, optimization, evaluations, and experimentation.
- [Comparison: TensorZero vs. LiteLLM](https://www.tensorzero.com/docs/comparison/litellm.md): TensorZero is an open-source alternative to LiteLLM featuring an LLM gateway, observability, optimization, evaluations, and experimentation.
- [Comparison: TensorZero vs. OpenPipe](https://www.tensorzero.com/docs/comparison/openpipe.md): TensorZero is an open-source alternative to OpenPipe featuring an LLM gateway, observability, optimization, evaluations, and experimentation.
- [Comparison: TensorZero vs. OpenRouter](https://www.tensorzero.com/docs/comparison/openrouter.md): TensorZero is an open-source alternative to OpenRouter featuring an LLM gateway, observability, optimization, evaluations, and experimentation.
- [Comparison: TensorZero vs. Portkey](https://www.tensorzero.com/docs/comparison/portkey.md): TensorZero is an open-source alternative to Portkey featuring an LLM gateway, observability, optimization, evaluations, and experimentation.
- [Deploy ClickHouse (optional)](https://www.tensorzero.com/docs/deployment/clickhouse.md): Learn how to deploy ClickHouse for TensorZero's observability features.
- [Optimize latency and throughput](https://www.tensorzero.com/docs/deployment/optimize-latency-and-throughput.md): Learn how to optimize the performance of the TensorZero Gateway for lower latency and higher throughput.
- [Deploy Postgres (optional)](https://www.tensorzero.com/docs/deployment/postgres.md): Learn how to deploy Postgres for advanced TensorZero features.
- [Set up TensorZero Autopilot](https://www.tensorzero.com/docs/deployment/tensorzero-autopilot.md): Learn how to set up TensorZero Autopilot on your self-hosted TensorZero deployme.
- [Deploy the TensorZero Gateway](https://www.tensorzero.com/docs/deployment/tensorzero-gateway.md): Learn how to deploy and customize the TensorZero Gateway.
- [Deploy the TensorZero UI](https://www.tensorzero.com/docs/deployment/tensorzero-ui.md): Learn how to deploy and customize the TensorZero UI.
- [Deploy Valkey / Redis (optional)](https://www.tensorzero.com/docs/deployment/valkey-redis.md): Learn how to deploy Valkey for high-performance rate limiting in TensorZero.
- [TensorZero Evaluations Overview](https://www.tensorzero.com/docs/evaluations/index.md): Learn how to use the TensorZero Evaluations to build principled LLM-powered applications.
- [CLI Reference](https://www.tensorzero.com/docs/evaluations/inference-evaluations/cli-reference.md): Learn how to use the TensorZero Evaluations CLI.
- [Configuration Reference](https://www.tensorzero.com/docs/evaluations/inference-evaluations/configuration-reference.md): Learn how to configure TensorZero Evaluations.
- [Tutorial: Inference Evaluations](https://www.tensorzero.com/docs/evaluations/inference-evaluations/tutorial.md): Learn how to use the TensorZero Inference Evaluations to build principled LLM-powered applications.
- [API Reference: Workflow Evaluations](https://www.tensorzero.com/docs/evaluations/workflow-evaluations/api-reference.md): API reference for workflow evaluations in TensorZero.
- [Tutorial: Workflow Evaluations](https://www.tensorzero.com/docs/evaluations/workflow-evaluations/tutorial.md): Learn how to use the TensorZero Workflow Evaluations to build principled LLM-powered applications.
- [Run adaptive A/B tests](https://www.tensorzero.com/docs/experimentation/run-adaptive-ab-tests.md): Learn how to use experimentation to test and iterate on your LLM applications with confidence.
- [Run static A/B tests](https://www.tensorzero.com/docs/experimentation/run-static-ab-tests.md): Learn how to use experimentation to test and iterate on your LLM applications with confidence.
- [Frequently Asked Questions](https://www.tensorzero.com/docs/faq.md): Learn more about TensorZero: how it works, why we built it, and more.
- [API Reference: Batch Inference](https://www.tensorzero.com/docs/gateway/api-reference/batch-inference.md): API reference for the Batch Inference endpoints.
- [API Reference: Datasets & Datapoints](https://www.tensorzero.com/docs/gateway/api-reference/datasets-datapoints.md): API reference for endpoints that manage datasets and datapoints.
- [API Reference: Feedback](https://www.tensorzero.com/docs/gateway/api-reference/feedback.md): API reference for the `/feedback` endpoint.
- [API Reference: Inference](https://www.tensorzero.com/docs/gateway/api-reference/inference.md): API reference for the `/inference` endpoint.
- [API Reference: Inference (OpenAI-Compatible)](https://www.tensorzero.com/docs/gateway/api-reference/inference-openai-compatible.md): API reference for the `/openai/v1/chat/completions` endpoint.
- [Benchmarks](https://www.tensorzero.com/docs/gateway/benchmarks.md): Benchmarks for the TensorZero Gateway: sub-millisecond latency overhead under extreme load
- [How to call any LLM](https://www.tensorzero.com/docs/gateway/call-any-llm.md): Learn how to call any LLM with a unified API using the TensorZero Gateway.
- [How to call the OpenAI Responses API](https://www.tensorzero.com/docs/gateway/call-the-openai-responses-api.md): Learn how to use OpenAI's Responses API with built-in tools like web search.
- [TensorZero Gateway Clients](https://www.tensorzero.com/docs/gateway/clients.md): The TensorZero Gateway can be used with the TensorZero Python client, with OpenAI clients (e.g. Python/Node), or via its HTTP API in any programming language.
- [Configuration Reference](https://www.tensorzero.com/docs/gateway/configuration-reference.md): Learn how to configure the TensorZero Gateway.
- [How to configure functions & variants](https://www.tensorzero.com/docs/gateway/configure-functions-and-variants.md): Learn how to configure functions and variants to define your LLM application logic with TensorZero.
- [How to configure models & providers](https://www.tensorzero.com/docs/gateway/configure-models-and-providers.md): Learn how to configure models and model providers to access LLMs with TensorZero.
- [How to create a prompt template](https://www.tensorzero.com/docs/gateway/create-a-prompt-template.md): Learn how to use prompt templates and schemas to manage complexity in your prompts.
- [Data Model](https://www.tensorzero.com/docs/gateway/data-model.md): Learn more about the data model used by TensorZero.
- [How to generate embeddings](https://www.tensorzero.com/docs/gateway/generate-embeddings.md): Learn how to generate embeddings from many model providers using the TensorZero Gateway with a unified API.
- [How to generate structured outputs](https://www.tensorzero.com/docs/gateway/generate-structured-outputs.md): Learn how to generate structured outputs (JSON) effectively using TensorZero.
- [Batch Inference](https://www.tensorzero.com/docs/gateway/guides/batch-inference.md): Learn how to process multiple requests at once with batch inference to save on inference costs at the expense of longer wait times.
- [Episodes](https://www.tensorzero.com/docs/gateway/guides/episodes.md): Learn how to use episodes to manage sequences of inferences that share a common outcome.
- [Inference Caching](https://www.tensorzero.com/docs/gateway/guides/inference-caching.md): Learn how to use inference caching with TensorZero Gateway.
- [Inference-Time Optimizations](https://www.tensorzero.com/docs/gateway/guides/inference-time-optimizations.md): Learn how to use inference-time strategies like dynamic in-context learning (DICL) and best-of-N sampling to optimize LLM performance.
- [Metrics & Feedback](https://www.tensorzero.com/docs/gateway/guides/metrics-feedback.md): Learn how to collect metrics and feedback about inferences or sequences of inferences.
- [Multimodal Inference](https://www.tensorzero.com/docs/gateway/guides/multimodal-inference.md): Learn how to use multimodal inference with TensorZero Gateway.
- [Retries & Fallbacks](https://www.tensorzero.com/docs/gateway/guides/retries-fallbacks.md): Learn how to use retries and fallbacks to handle errors and improve reliability with TensorZero.
- [Streaming Inference](https://www.tensorzero.com/docs/gateway/guides/streaming-inference.md): Learn how to use streaming inference with TensorZero Gateway.
- [Tool Use (Function Calling)](https://www.tensorzero.com/docs/gateway/guides/tool-use.md): Learn how to use tool use (function calling) with TensorZero Gateway.
- [Overview](https://www.tensorzero.com/docs/gateway/index.md): The TensorZero Gateway is a high-performance model gateway that provides a unified interface for all your LLM applications.
- [Overview](https://www.tensorzero.com/docs/index.md): TensorZero is an open-source stack for industrial-grade LLM applications that unifies an LLM gateway, observability, optimization, evaluation, and experimentation.
- [Getting Started with Anthropic](https://www.tensorzero.com/docs/integrations/model-providers/anthropic.md): Learn how to use TensorZero with Anthropic LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with AWS Bedrock](https://www.tensorzero.com/docs/integrations/model-providers/aws-bedrock.md): Learn how to use TensorZero with AWS Bedrock LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with AWS SageMaker](https://www.tensorzero.com/docs/integrations/model-providers/aws-sagemaker.md): Learn how to use TensorZero with AWS SageMaker LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with Azure](https://www.tensorzero.com/docs/integrations/model-providers/azure.md): Learn how to use TensorZero with Azure LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with DeepSeek](https://www.tensorzero.com/docs/integrations/model-providers/deepseek.md): Learn how to use TensorZero with DeepSeek LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with Fireworks AI](https://www.tensorzero.com/docs/integrations/model-providers/fireworks.md): Learn how to use TensorZero with Fireworks AI LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with GCP Vertex AI Anthropic](https://www.tensorzero.com/docs/integrations/model-providers/gcp-vertex-ai-anthropic.md): Learn how to use TensorZero with GCP Vertex AI Anthropic LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Infererence with GCP Vertex AI Gemini](https://www.tensorzero.com/docs/integrations/model-providers/gcp-vertex-ai-gemini.md): Learn how to use TensorZero with GCP Vertex AI Gemini LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with Google AI Studio (Gemini API)](https://www.tensorzero.com/docs/integrations/model-providers/google-ai-studio-gemini.md): Learn how to use TensorZero with Google AI Studio LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with Groq](https://www.tensorzero.com/docs/integrations/model-providers/groq.md): Learn how to use TensorZero with Groq LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with Hyperbolic](https://www.tensorzero.com/docs/integrations/model-providers/hyperbolic.md): Learn how to use TensorZero with Hyperbolic LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Overview](https://www.tensorzero.com/docs/integrations/model-providers/index.md): The TensorZero Gateway integrates with the major LLM providers.
- [Getting Started with Mistral](https://www.tensorzero.com/docs/integrations/model-providers/mistral.md): Learn how to use TensorZero with Mistral LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with OpenAI](https://www.tensorzero.com/docs/integrations/model-providers/openai.md): Learn how to use TensorZero with OpenAI LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with OpenAI-Compatible Endpoints (e.g. Ollama)](https://www.tensorzero.com/docs/integrations/model-providers/openai-compatible.md): Learn how to use TensorZero with OpenAI-compatible LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with OpenRouter](https://www.tensorzero.com/docs/integrations/model-providers/openrouter.md): Learn how to use TensorZero with OpenRouter LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with SGLang](https://www.tensorzero.com/docs/integrations/model-providers/sglang.md): Learn how to use TensorZero with self-hosted SGLang LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with Text Generation Inference (TGI)](https://www.tensorzero.com/docs/integrations/model-providers/tgi.md): Learn how to use TensorZero with self-hosted HuggingFace TGI LLMs: open-source gateway, observability, optimization, evaluations, experimentation.
- [Getting Started with Together AI](https://www.tensorzero.com/docs/integrations/model-providers/together.md): Learn how to use TensorZero with Together AI LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with vLLM](https://www.tensorzero.com/docs/integrations/model-providers/vllm.md): Learn how to use TensorZero with self-hosted vLLM LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [Getting Started with xAI (Grok)](https://www.tensorzero.com/docs/integrations/model-providers/xai.md): Learn how to use TensorZero with xAI (Grok) LLMs: open-source gateway, observability, optimization, evaluations, and experimentation.
- [How to query historical inferences](https://www.tensorzero.com/docs/observability/query-historical-inferences.md): Learn how to retrieve and filter historical inferences from the TensorZero Gateway.
- [Centralize auth, rate limits, and more](https://www.tensorzero.com/docs/operations/centralize-auth-rate-limits-and-more.md): Learn how to use gateway relay to centralize auth, rate limits, and credentials while letting teams manage their own TensorZero deployments.
- [Enforce custom rate limits](https://www.tensorzero.com/docs/operations/enforce-custom-rate-limits.md): Learn how to set up granular custom rate limits for your TensorZero Gateway.
- [Export OpenTelemetry traces (OTLP)](https://www.tensorzero.com/docs/operations/export-opentelemetry-traces.md): Learn how to export traces from the TensorZero Gateway to an external OpenTelemetry-compatible observability system.
- [Export Prometheus metrics](https://www.tensorzero.com/docs/operations/export-prometheus-metrics.md): Learn how the TensorZero Gateway exports Prometheus-compatible metrics for monitoring and debugging.
- [Extend TensorZero](https://www.tensorzero.com/docs/operations/extend-tensorzero.md): Learn how to extend or override TensorZero to access provider features we don't support out of the box.
- [Manage credentials (API keys)](https://www.tensorzero.com/docs/operations/manage-credentials.md): Learn how to manage credentials (API keys) in TensorZero.
- [Organize your configuration](https://www.tensorzero.com/docs/operations/organize-your-configuration.md): Learn best practices for organizing your configuration as your project grows in complexity.
- [Set up auth for TensorZero](https://www.tensorzero.com/docs/operations/set-up-auth-for-tensorzero.md): Learn how to set up TensorZero API keys to authenticate your inference requests and manage access control for your workflows.
- [Dynamic In-Context Learning (DICL)](https://www.tensorzero.com/docs/optimization/dynamic-in-context-learning-dicl.md): Learn how to use Dynamic In-Context Learning to optimize your LLM applications.
- [GEPA](https://www.tensorzero.com/docs/optimization/gepa.md): Learn how to use automated prompt engineering to optimize your LLM applications.
- [Overview](https://www.tensorzero.com/docs/optimization/index.md): Learn more about using TensorZero Recipes to optimize your LLM applications.
- [Quickstart](https://www.tensorzero.com/docs/quickstart.md): Get up and running with TensorZero in 5 minutes.
- [Vision & Roadmap](https://www.tensorzero.com/docs/vision-and-roadmap.md): Learn more about TensorZero's vision and roadmap.


## Optional

- [Documentation](https://www.tensorzero.com/docs)
- [Blog](https://www.tensorzero.com/blog)
- [Slack](https://www.tensorzero.com/slack)
- [Discord](https://www.tensorzero.com/discord)