Skip to main content
This page shows how to:
  • Use a unified API. TensorZero provides the same chat completion format for the Responses API.
  • Access built-in tools. Enable built-in tools from OpenAI like web_search.
  • Enable reasoning models. Support models with extended thinking capabilities.
You can find a complete runnable example of this guide on GitHub.

​
Call the OpenAI Responses API

The TensorZero Python SDK provides a unified API for calling OpenAI’s Responses API.
1

Set up your OpenAI API key

You can set the OPENAI_API_KEY environment variable with your API key.
export OPENAI_API_KEY="sk-..."
2

Install the TensorZero Python SDK

You can install the TensorZero SDK with a Python package manager like pip.
pip install tensorzero
3

Configure a model for the OpenAI Responses API

Create a configuration file with a model using api_type = "responses" and provider tools:
tensorzero.toml
[models.gpt-5-mini-responses-web-search]
routing = ["openai"]

[models.gpt-5-mini-responses-web-search.providers.openai]
type = "openai"
model_name = "gpt-5-mini"
api_type = "responses"
include_encrypted_reasoning = true
provider_tools = [{type = "web_search"}]  # built-in OpenAI web search tool
# Enable plain-text summaries of encrypted reasoning
extra_body = [
    { pointer = "/reasoning", value = { effort = "low", summary = "auto" } }
]
If you don’t need to customize the model configuration (e.g. include_encrypted_reasoning, provider_tools), you can use the short-hand model name openai::responses::gpt-5-codex to call it directly.
4

Deploy a standalone (HTTP) TensorZero Gateway

Let’s deploy a standalone TensorZero Gateway using Docker. For simplicity, we’ll use the gateway with the configuration above.
docker run \
  -e OPENAI_API_KEY \
  -v $(pwd)/tensorzero.toml:/app/config/tensorzero.toml:ro \
  -p 3000:3000 \
  tensorzero/gateway \
  --config-file /app/config/tensorzero.toml
See the TensorZero Gateway Deployment page for more details.
5

Initialize the TensorZero Gateway client

Let’s initialize the TensorZero Gateway client and point it to the gateway we just launched.
from tensorzero import TensorZeroGateway

t0 = TensorZeroGateway.build_http(gateway_url="http://localhost:3000")
The TensorZero Python SDK includes a synchronous TensorZeroGateway client and an asynchronous AsyncTensorZeroGateway client. Both options support running the gateway embedded in your application with build_embedded or connecting to a standalone gateway with build_http. See Clients for more details.
6

Call the LLM

OpenAI web search can take up to a minute to complete.
response = t0.inference(
    model_name="gpt-5-mini-responses-web-search",
    input={
        "messages": [
            {
                "role": "user",
                "content": "What is the current population of Japan?",
            }
        ]
    },
    # Thought summaries are enabled in tensorzero.toml via extra_body
)

ChatInferenceResponse(
    inference_id=UUID('0199ff78-6246-7c12-b4b0-6e3a881cc6b9'),
    episode_id=UUID('0199ff78-6246-7c12-b4b0-6e4367f949b8'),
    variant_name='gpt-5-mini-responses-web-search',
    content=[
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo9...',
            summary=[
                ThoughtSummaryBlock(
                    text="I need to search for Japan's current population data.",
                    type='summary_text'
                )
            ],
            provider_type='openai'
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fda57d481969c3603df0d675348',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': 'Japan population 2025 October 2025 population estimate Statistics Bureau of Japan'
                }
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fdf9b988196b36756d639e2b015',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "Ministry of Internal Affairs and Communications Japan population Oct 1 2024 'total population' 'Japan' 'population estimates' '2024' 'Oct. 1' '総人口' '令和6年' "
                }
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe1a388819684971acfdaf4cd44',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "Ministry of Internal Affairs and Communications population Japan Oct 1 2024 total population 'Oct. 1, 2024' 'population' 'Japan' 'MIC' 'population estimates' '2024' '総人口' "
                }
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe439788196911a195c70cc8ca9',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {'type': 'search'}
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe6b140819690a4468d3304fece',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {'type': 'search'}
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe81e408196921b69174f6abaf7',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {'type': 'search'}
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59feda6188196827a0b5aa01e96a1',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "United Nations World Population Prospects 2024 Japan 2025 population 'Japan population 2025' 'World Population Prospects 2024' 'Japan' "
                }
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59ff3cc8881968d1c5c9c1bbe4ecc',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "UN World Population Prospects 2024 Japan population 2025 '123,103,479' 'Japan 2025' 'World Population Prospects' 'Japan' '2025' "
                }
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59ff67ed48196a0054a38e96f8e0c',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "United Nations population Japan 2025 'World Population Prospects 2024' 'Japan population 2025' site:un.org"
                }
            },
            model_name='gpt-5-mini-responses-web-search',
            provider_name='openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            provider_type=None
        ),
        Text(
            text="Short answer: about 123–124 million people.\n\nMore precisely:\n- Japan's official estimate (Ministry of Internal Affairs and Communications / e‑Stat) reported a total population of 123,802,000 (including foreign residents) as of October 1, 2024 (release published Apr 14, 2025). ([e-stat.go.jp](https://www.e-stat.go.jp/en/stat-search/files?layout=dataset&page=1&query=Population+Estimates%2C+natural))  \n- The United Nations (WPP 2024, used by sources such as Worldometer) gives a mid‑2025 estimate of about 123.1 million. ([srv1.worldometers.info](https://srv1.worldometers.info/world-population/japan-population/?utm_source=openai))\n\nDo you want a live "right now" estimate for today (Oct 20, 2025) or a breakdown by Japanese nationals vs. foreign residents? I can fetch the latest live or official figures for the exact date you want.",
            arguments=None,
            type='text'
        )
    ],
    usage=Usage(input_tokens=29904, output_tokens=1921),
    finish_reason=None,
    original_response=None
)

​
Call the OpenAI Responses API with Azure

You can call the OpenAI Responses API with Azure by setting api_base in your configuration to your Azure deployment URL. 
[models.azure-gpt-5-mini-responses]
routing = ["azure"]

[models.azure-gpt-5-mini-responses.providers.azure]
type = "openai"  # CAREFUL: not `azure`!
api_base = "https://YOUR-DEPLOYMENT-HERE.openai.azure.com/openai/v1/"  # TODO: Insert your API base URL here
api_key_location = "env::AZURE_API_KEY"
model_name = "gpt-5-mini"
api_type = "responses"
The azure model provider does not support the Responses API. You must use the openai provider with a custom api_base instead.