Tokens/Second, TTFT Latency & API Cost Efficiency
Inference Speed, Latency & Price Frontier
Benchmarking real-world generation throughput (tokens/sec), time-to-first-token (TTFT ms), and pricing per million tokens across provider APIs and open-weight hostings.
Human BaselineN/A (Hardware & Quantization Metric)
Contamination StatusZero (Empirical runtime telemetry)
Top Frontier ModelGemini 3.7 Flash (182 tok/s at $0.75/1M)
Why Classic Benchmarks Failed
Static model scoreboards ignored deployment latency, making models with high latency impractical for real-time agentic tool loops.
The TrustTheBench & LiveBench Approach
Artificial Analysis continuous API probing across 20+ cloud providers under standardized load conditions.
Verified Category Leaderboard
Ranked strictly by verified Inference Speed, Latency & Price Frontier performance.
| Rank | Model | Research Lab | Access Type | Category Score | Speed | Pricing / 1M |
|---|---|---|---|---|---|---|
| #1 | OpenAI Decisions API | OpenAI | proprietary | 320 tok/s | 320 tok/s | $0.35 in |
| #2 | Jev 1.14 | TypeSafe AI | proprietary | 290 tok/s | 290 tok/s | $0.25 in |
| #3 | Jev | TypeSafe AI | proprietary | 280 tok/s | 280 tok/s | $0.20 in |
| #4 | Microsoft-Decision-1 | Microsoft Research & Azure | open-weight | 240 tok/s | 240 tok/s | $0.08 in |
| #5 | Amazon Nova Micro 1.0 | Amazon AWS Bedrock | proprietary | 240 tok/s | 240 tok/s | $0.04 in |
| #6 | Gemini 3.5 Flash-Lite | Google DeepMind | proprietary | 220 tok/s | 220 tok/s | $0.10 in |
| #7 | IndicTrans-2 | AI4Bharat | open-weight | 220 tok/s | 220 tok/s | $0.02 in |
| #8 | Amazon Nova Lite | Amazon AWS Bedrock | proprietary | 200 tok/s | 200 tok/s | $0.06 in |
| #9 | Phi-4-mini | Microsoft Research & Azure | open-weight | 195 tok/s | 195 tok/s | $0.02 in |
| #10 | DeepSeek V4-Flash | DeepSeek | open-weight | 190 tok/s | 190 tok/s | $0.14 in |
| #11 | Grok 3 Mini | xAI | proprietary | 190 tok/s | 190 tok/s | $0.30 in |
| #12 | Gemini 3.8 Flash | Google DeepMind | proprietary | 185 tok/s | 185 tok/s | $0.75 in |
| #13 | Gemini 2.0 Flash | Google DeepMind | proprietary | 185 tok/s | 185 tok/s | $0.10 in |
| #14 | Gemini 3.7 Flash | Google DeepMind | proprietary | 182 tok/s | 182 tok/s | $0.75 in |
| #15 | Saaras v3 (Speech-to-Text) | Sarvam AI | proprietary | 180 tok/s | 180 tok/s | $0.10 in |
Deployment Recommendations
Best Overall Frontier
Gemini 3.7 Flash (182 tok/s at $0.75/1M)
Maximum reasoning depth and lowest error rate on difficult boundary problems.
Best Open-Weight
DeepSeek V4-Flash (190 tok/s at $0.14/1M)
Host locally or via cost-effective cloud providers with full data privacy.
Best Value & Speed
Gemini 3.5 Flash-Lite (220 tok/s at $0.10/1M)
Optimal price-to-performance ratio for high-volume automated agent pipelines.