TensorZero

TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products.

1. Integrate our model gateway
2. Send metrics or feedback
3. Unlock compounding improvements in quality, cost, and latency

It enables a data & learning flywheel for LLMs by unifying:

• Inference: one API for all LLMs, with <1ms P99 overhead
• Observability: inference & feedback → your database
• Optimization: better prompts, models, inference strategies
• Experimentation: built-in A/B testing, routing, fallbacks

Read more →

Who are we?

We’re a team of two based in NYC.

Viraj Mehta (CTO) recently completed his PhD from CMU, with an emphasis on reinforcement learning for LLMs and nuclear fusion, and previously worked in machine learning at KKR and a fintech startup; he holds a BS in math and an MS in computer science from Stanford.

Gabriel Bianconi (CEO) was the chief product officer at Ondo Finance ($14B+ valuation in 2024) and previously spent years consulting on machine learning for companies ranging from early-stage tech startups to some of the largest financial firms; he holds BS and MS degrees in computer science from Stanford.

Get started

Start building today. Check out our Github, Quick Start (5min), or Tutorial (30min).

Questions? Ask us on Slack or Discord.

Using TensorZero at work? Email us at hello@tensorzero.com to set up a Slack or Teams channel with your team (free).