TensorZero

TensorZero is an open-source platform that enables LLM applications that learn from real-world experience.

1. Integrate our model gateway
2. Send metrics or feedback
3. Optimize prompts, models, and inference-time strategies
4. Unlock compounding improvements in quality, cost, and latency

It provides a data & learning flywheel for LLMs by unifying:

• Inference: one API for all LLMs, with <1ms P99 overhead
• Observability: inference & feedback → your database
• Optimization: from prompts to fine-tuning and RL (& even 🍓? →)
• Experimentation: built-in A/B testing, routing, fallbacks

Read more →

Who are we?

We’re a small team based in NYC.

Viraj Mehta (CTO) recently completed his PhD from CMU, with an emphasis on reinforcement learning for LLMs and nuclear fusion, and previously worked in machine learning at KKR and a fintech startup; he holds a BS in math and an MS in computer science from Stanford.

Gabriel Bianconi (CEO) was the chief product officer at Ondo Finance ($14B+ valuation in 2024) and previously spent years consulting on machine learning for companies ranging from early-stage tech startups to some of the largest financial firms; he holds BS and MS degrees in computer science from Stanford.

Get started

Start building today. Check out our Github, Quick Start, or Tutorial.

Questions? Ask us on Slack or Discord.

Using TensorZero at work? Email us at hello@tensorzero.com to set up a Slack or Teams channel with your team (free).