ReasoningActive
Trust Score:
94
LiveBench (Continuous)
Monthly refreshed benchmark designed to prevent test-set contamination across reasoning, coding, mathematics, data analysis, and instruction following.
Launched: 2024-06-01Refresh: monthly
Performance Timeline
Longitudinal progression of model scores against human baselines.Performance & Historical Trajectory
Empirical score progression across model release dates and evaluation rounds.
Independent Vendor-Reported Human Baseline (68.5%)
| Model | Score | Date | Source Type | Provenance |
|---|---|---|---|---|
| Claude Fable 5 | 83% | 2025-02-01 | independent | Source ↗ |
| GPT-5.6 Sol | 82.8% | 2025-02-01 | independent | Source ↗ |
| Claude Opus 5 | 80.1% | 2025-02-01 | independent | Source ↗ |
| Claude Fable 5.1 (Hybrid Reasoning) | 84.7% | 2025-02-01 | independent | Source ↗ |
| GPT-6 Astra (max) | 84.2% | 2025-02-01 | independent | Source ↗ |
| GPT-5.5 Thinking | 80.2% | 2025-02-01 | independent | Source ↗ |
| GPT-5 | 82% | 2025-02-01 | independent | Source ↗ |
| Kimi K3 | 82% | 2025-02-01 | independent | Source ↗ |
| Grok 4 | 81.2% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4.1 Flash | 83.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.7 Flash | 78.8% | 2025-02-01 | independent | Source ↗ |
| Grok 4.6 | 78.5% | 2025-02-01 | independent | Source ↗ |
| Claude Sonnet 5 | 80.4% | 2025-02-01 | independent | Source ↗ |
| Gemini 3 Pro | 79.8% | 2025-02-01 | independent | Source ↗ |
| Claude 3.7 Sonnet | 81.5% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4-Pro | 77.9% | 2025-02-01 | independent | Source ↗ |
| o3 | 82.5% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.6 Flash | 77.2% | 2025-02-01 | independent | Source ↗ |
| Qwen3-235B | 77.4% | 2025-02-01 | independent | Source ↗ |
| Claude Opus 5.5 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-6 Astra Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Opus 5 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.6 Terra Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Gemini 3 Pro Image (Nano Banana Pro) | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Opus 4.8 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.5 Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Opus 4.7 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.4 Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.1 Pro Custom Tools | 80.5% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.1 Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Opus 4.6 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.2 Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Opus 4.5 | 80.5% | 2025-02-01 | independent | Source ↗ |
| Gemini 3 Pro Image (Nano Banana Pro) | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5 Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude 4.1 Opus | 80.5% | 2025-02-01 | independent | Source ↗ |
| o3 Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| o1-pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.5 Pro | 81% | 2025-02-01 | independent | Source ↗ |
| Nemotron 3 Ultra 550B | 78.5% | 2025-02-01 | independent | Source ↗ |
| DeepSeek-R1 | 80.8% | 2025-02-01 | independent | Source ↗ |
| QwQ Plus | 76.5% | 2025-02-01 | independent | Source ↗ |
| Amazon Nova Premier 1.0 | 78.2% | 2025-02-01 | independent | Source ↗ |
| GLM-5.2 | 77% | 2025-02-01 | independent | Source ↗ |
| Llama 4 Maverick | 77.1% | 2025-02-01 | independent | Source ↗ |
| Sonar Reasoning Pro | 80.5% | 2025-02-01 | independent | Source ↗ |
| Ollama DeepSeek-R1 (Q4_K_M) | 80% | 2025-02-01 | independent | Source ↗ |
| Grok 3 | 81.2% | 2025-02-01 | independent | Source ↗ |
| o3-mini | 80.2% | 2025-02-01 | independent | Source ↗ |
| MiniMax M3 | 76.8% | 2025-02-01 | independent | Source ↗ |
| Llama 4 Scout | 76% | 2025-02-01 | independent | Source ↗ |
| Sarvam 105B | 78.4% | 2025-02-01 | independent | Source ↗ |
| GPT-6.1 Sol Pro | 77.9% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5.3 Prime | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-6 Sol Pro | 77.9% | 2025-02-01 | independent | Source ↗ |
| Xiaomi: MiMo-V2.6-Pro-UltraSpeed | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5.6 Sol Pro | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5.1-Codex-Max | 77.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.5 Pro | 77.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.5 Pro 06-05 | 77.9% | 2025-02-01 | independent | Source ↗ |
| Amazon: Nova Pro 1.0 | 77.9% | 2025-02-01 | independent | Source ↗ |
| Mistral Large | 77.9% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4-Flash | 74.5% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.0 Flash Thinking | 74.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.5 Flash-Lite | 73.8% | 2025-02-01 | independent | Source ↗ |
| o1 | 73.5% | 2025-02-01 | independent | Source ↗ |
| GPT-4.5 | 71% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.0 Pro | 72.8% | 2025-02-01 | independent | Source ↗ |
| QwQ 32B | 73.8% | 2025-02-01 | independent | Source ↗ |
| Mistral Large 3 | 72% | 2025-02-01 | independent | Source ↗ |
| Qwen 2.5 Max | 71.4% | 2025-02-01 | independent | Source ↗ |
| Llama-3.1-Nemotron-70B-Instruct | 75.4% | 2025-02-01 | independent | Source ↗ |
| Qwen 2.5 72B Instruct | 76.5% | 2025-02-01 | independent | Source ↗ |
| Amazon Nova Pro 1.0 | 74% | 2025-02-01 | independent | Source ↗ |
| Nemotron 3 Super 120B | 74.2% | 2025-02-01 | independent | Source ↗ |
| Mistral Large 4 | 74.8% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4 Pro 0813 | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-5.6 Luna Pro | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-5.6 Luna | 74.8% | 2025-02-01 | independent | Source ↗ |
| Nemotron 3 Ultra | 74.8% | 2025-02-01 | independent | Source ↗ |
| Mistral Large 3 2512 | 74.8% | 2025-02-01 | independent | Source ↗ |
| Phi-4 | 76.5% | 2025-02-01 | independent | Source ↗ |
| Claude 3.5 Sonnet | 69.8% | 2025-02-01 | independent | Source ↗ |
| Hunyuan-Large | 74.8% | 2025-02-01 | independent | Source ↗ |
| Nemotron-4-340B-Instruct | 73% | 2025-02-01 | independent | Source ↗ |
| Llama 3.3 70B Instruct | 75.8% | 2025-02-01 | independent | Source ↗ |
| Gemma 3 27B | 76.2% | 2025-02-01 | independent | Source ↗ |
| Sarvam 30B | 74% | 2025-02-01 | independent | Source ↗ |
| Mistral Large 2 | 75.2% | 2025-02-01 | independent | Source ↗ |
| Kimi k1.5 | 70.5% | 2025-02-01 | independent | Source ↗ |
| Sonar Pro | 77% | 2025-02-01 | independent | Source ↗ |
| Ollama Llama 3.3 (Q4_K_M) | 75% | 2025-02-01 | independent | Source ↗ |
| Yi-Lightning | 73.5% | 2025-02-01 | independent | Source ↗ |
| Amazon Nova Pro | 74.2% | 2025-02-01 | independent | Source ↗ |
| DeepSeek-V3 | 67.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.0 Flash | 67.5% | 2025-02-01 | independent | Source ↗ |
| Llama 3.1 405B | 66.8% | 2025-02-01 | independent | Source ↗ |
| MiniMax-Text-01 | 73% | 2025-02-01 | independent | Source ↗ |
| WizardLM-2 8x22B | 71.5% | 2025-02-01 | independent | Source ↗ |
| Amazon Nova 2 Lite | 72.8% | 2025-02-01 | independent | Source ↗ |
| Qwen 2.5 Coder 32B | 67% | 2025-02-01 | independent | Source ↗ |
| ERNIE 4.0 Turbo | 71.8% | 2025-02-01 | independent | Source ↗ |
| GPT-4o | 66.4% | 2025-02-01 | independent | Source ↗ |
| GLM-4-Plus | 72.5% | 2025-02-01 | independent | Source ↗ |
| GPT-6 Luna Pro | 71.3% | 2025-02-01 | independent | Source ↗ |
| GPT-6 Luna | 71.3% | 2025-02-01 | independent | Source ↗ |
| Xiaomi: MiMo-V2.6-Pro | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4 Pro 0423 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Xiaomi: MiMo-V2.5-Pro | 71.3% | 2025-02-01 | independent | Source ↗ |
| Llama 3.3 70B | 65.2% | 2025-02-01 | independent | Source ↗ |
| Phi-4-multimodal | 72% | 2025-02-01 | independent | Source ↗ |
| Reka Core | 69.2% | 2025-02-01 | independent | Source ↗ |
| Mistral Large 2 | 63.4% | 2025-02-01 | independent | Source ↗ |
| Gemma 2 27B | 71.5% | 2025-02-01 | independent | Source ↗ |
| Solar Pro | 68% | 2025-02-01 | independent | Source ↗ |
| Gemini 1.5 Pro | 64.1% | 2025-02-01 | independent | Source ↗ |
| DBRX Instruct | 67.2% | 2025-02-01 | independent | Source ↗ |
| Lyria 3 Pro | 67.8% | 2025-02-01 | independent | Source ↗ |
| Snowflake Arctic | 66.5% | 2025-02-01 | independent | Source ↗ |
| Amazon Nova Lite | 69.5% | 2025-02-01 | independent | Source ↗ |
| GPT-4o mini | 56.2% | 2025-02-01 | independent | Source ↗ |
| GPT-6 Astra | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Fable 5.1 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.6 Terra | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Fable 5 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT Chat Latest | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.5 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.4 Image 2 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.4 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.3-Codex | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Sonnet 4.6 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.2-Codex | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.2 Chat | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-5.2 | 80.5% | 2025-02-01 | independent | Source ↗ |
| Amazon: Nova Premier 1.0 | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude Sonnet 4.5 | 80.5% | 2025-02-01 | independent | Source ↗ |
| Claude 4 Sonnet | 80.5% | 2025-02-01 | independent | Source ↗ |
| o1 | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-4o | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-4 Turbo | 80.5% | 2025-02-01 | independent | Source ↗ |
| GPT-4 | 80.5% | 2025-02-01 | independent | Source ↗ |
| QwQ-32B | 79.2% | 2025-02-01 | independent | Source ↗ |
| Gemini 4 Argon | 77.9% | 2025-02-01 | independent | Source ↗ |
| Nano Banana 2.1 | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-6.1 Sol | 77.9% | 2025-02-01 | independent | Source ↗ |
| Claude Sonnet 5.5 | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-6 Sol | 77.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.8 Flash | 77.9% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5.3 | 77.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.7 Flash | 77.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.6 Flash | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5.6 Sol | 77.9% | 2025-02-01 | independent | Source ↗ |
| Claude Sonnet 5 | 77.9% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5.2 | 77.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.5 Flash | 77.9% | 2025-02-01 | independent | Source ↗ |
| Mistral Medium 3.5 | 77.9% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5.1 | 77.9% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5V Turbo | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5.4 Mini | 77.9% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5 Turbo | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT Audio | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5.1 | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5.1-Codex | 77.9% | 2025-02-01 | independent | Source ↗ |
| Claude Haiku 4.5 | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5 Image | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-5 | 77.9% | 2025-02-01 | independent | Source ↗ |
| o4 Mini High | 77.9% | 2025-02-01 | independent | Source ↗ |
| o3 | 77.9% | 2025-02-01 | independent | Source ↗ |
| o4 Mini | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-4.1 | 77.9% | 2025-02-01 | independent | Source ↗ |
| o3 Mini High | 77.9% | 2025-02-01 | independent | Source ↗ |
| o3 Mini | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-4o | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-4o | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-4o | 77.9% | 2025-02-01 | independent | Source ↗ |
| Mixtral 8x22B Instruct | 77.9% | 2025-02-01 | independent | Source ↗ |
| GPT-3.5 Turbo 16k | 77.9% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.5 Flash (Thinking) | 79.8% | 2025-02-01 | independent | Source ↗ |
| Claude 3.5 Sonnet (v2) | 78.5% | 2025-02-01 | independent | Source ↗ |
| Nemotron-4-340B-Reward | 74.5% | 2025-02-01 | independent | Source ↗ |
| Command A+ | 74.8% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5.3 FlashX | 74.8% | 2025-02-01 | independent | Source ↗ |
| Hy4 | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.5 Flash Lite | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.1 Flash Lite Image | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.1 Flash Image | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.1 Flash Lite | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-5.4 Nano | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.1 Flash Lite | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 3.1 Flash Image | 74.8% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5 | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT Audio Mini | 74.8% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 4.7 | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 3 Flash | 74.8% | 2025-02-01 | independent | Source ↗ |
| Devstral 2 2512 | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-5.1-Codex-Mini | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-5 Image Mini | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.5 Flash Image | 74.8% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 4.6 | 74.8% | 2025-02-01 | independent | Source ↗ |
| Mistral Medium 3.1 | 74.8% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 4.5V | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-5 Mini | 74.8% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 4.5 | 74.8% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.5 Flash | 74.8% | 2025-02-01 | independent | Source ↗ |
| R1 0528 | 74.8% | 2025-02-01 | independent | Source ↗ |
| Mistral Medium 3 | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-4.1 Mini | 74.8% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V3 0324 | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-3.5 Turbo (older v0613) | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-3.5 Turbo Instruct | 74.8% | 2025-02-01 | independent | Source ↗ |
| GPT-3.5 Turbo | 74.8% | 2025-02-01 | independent | Source ↗ |
| Sarvam Vision (Document AI) | 73.2% | 2025-02-01 | independent | Source ↗ |
| Saaras v3 (Speech-to-Text) | 71.5% | 2025-02-01 | independent | Source ↗ |
| Sonar | 74% | 2025-02-01 | independent | Source ↗ |
| Claude Haiku 5.5 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Xiaomi: MiMo-V2.6-Flash | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4.1 Flash | 71.3% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 5.3 Flash | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4 Flash Vision Exp | 71.3% | 2025-02-01 | independent | Source ↗ |
| Hy-MT2-30B-A3B | 71.3% | 2025-02-01 | independent | Source ↗ |
| Hy-MT2-7B | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4 Flash 0731 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Hy3 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Hy3 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Xiaomi: MiMo-V2.5 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Gemma 4 26B A4B | 71.3% | 2025-02-01 | independent | Source ↗ |
| Gemma 4 31B | 71.3% | 2025-02-01 | independent | Source ↗ |
| Mistral Small 4 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Nemotron 3 Super | 71.3% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 4.7 Flash | 71.3% | 2025-02-01 | independent | Source ↗ |
| Nemotron 3 Nano 30B A3B | 71.3% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 4.6V | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V3.2 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Voxtral Small 24B 2507 | 71.3% | 2025-02-01 | independent | Source ↗ |
| gpt-oss-safeguard-20b | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V3.2 Exp | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V3.1 Terminus | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V3.1 | 71.3% | 2025-02-01 | independent | Source ↗ |
| GPT-5 Nano | 71.3% | 2025-02-01 | independent | Source ↗ |
| gpt-oss-120b | 71.3% | 2025-02-01 | independent | Source ↗ |
| Codestral 2508 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Z.ai: GLM 4.5 Air | 71.3% | 2025-02-01 | independent | Source ↗ |
| Gemini 2.5 Flash Lite | 71.3% | 2025-02-01 | independent | Source ↗ |
| Hunyuan A13B Instruct | 71.3% | 2025-02-01 | independent | Source ↗ |
| Mistral Small 3.2 24B | 71.3% | 2025-02-01 | independent | Source ↗ |
| GPT-4.1 Nano | 71.3% | 2025-02-01 | independent | Source ↗ |
| Llama 4 Maverick | 71.3% | 2025-02-01 | independent | Source ↗ |
| Llama 4 Scout | 71.3% | 2025-02-01 | independent | Source ↗ |
| Mistral Small 3.1 24B | 71.3% | 2025-02-01 | independent | Source ↗ |
| Gemma 3 27B | 71.3% | 2025-02-01 | independent | Source ↗ |
| Saba | 71.3% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V3 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Llama 3.3 70B Instruct | 71.3% | 2025-02-01 | independent | Source ↗ |
| Amazon: Nova Lite 1.0 | 71.3% | 2025-02-01 | independent | Source ↗ |
| Llama 3.2 1B Instruct | 71.3% | 2025-02-01 | independent | Source ↗ |
| Llama 3.2 3B Instruct | 71.3% | 2025-02-01 | independent | Source ↗ |
| Command R (08-2024) | 71.3% | 2025-02-01 | independent | Source ↗ |
| Llama 3.1 70B Instruct | 71.3% | 2025-02-01 | independent | Source ↗ |
| GPT-4o-mini | 71.3% | 2025-02-01 | independent | Source ↗ |
| GPT-4o-mini | 71.3% | 2025-02-01 | independent | Source ↗ |
| Gemma 2 27B | 71.3% | 2025-02-01 | independent | Source ↗ |
| Bulbul v3 (Text-to-Speech) | 70% | 2025-02-01 | independent | Source ↗ |
| Amazon Nova Lite 1.0 | 69.5% | 2025-02-01 | independent | Source ↗ |
| Switchyard | 67.8% | 2025-02-01 | independent | Source ↗ |
| Hy-MT2-1.8B | 67.8% | 2025-02-01 | independent | Source ↗ |
| Nemotron 3.5 Lightning | 67.8% | 2025-02-01 | independent | Source ↗ |
| Nemotron 3.5 Content Safety | 67.8% | 2025-02-01 | independent | Source ↗ |
| DeepSeek V4 Flash 0423 | 67.8% | 2025-02-01 | independent | Source ↗ |
| Lyria 3 Clip | 67.8% | 2025-02-01 | independent | Source ↗ |
| Ministral 3 14B 2512 | 67.8% | 2025-02-01 | independent | Source ↗ |
| Ministral 3 8B 2512 | 67.8% | 2025-02-01 | independent | Source ↗ |
| Ministral 3 3B 2512 | 67.8% | 2025-02-01 | independent | Source ↗ |
| gpt-oss-20b | 67.8% | 2025-02-01 | independent | Source ↗ |
| Llama Guard 4 12B | 67.8% | 2025-02-01 | independent | Source ↗ |
| Gemma 3 4B | 67.8% | 2025-02-01 | independent | Source ↗ |
| Gemma 3 12B | 67.8% | 2025-02-01 | independent | Source ↗ |
| Mistral Small 3 | 67.8% | 2025-02-01 | independent | Source ↗ |
| Phi 4 | 67.8% | 2025-02-01 | independent | Source ↗ |
| Command R7B (12-2024) | 67.8% | 2025-02-01 | independent | Source ↗ |
| Amazon: Nova Micro 1.0 | 67.8% | 2025-02-01 | independent | Source ↗ |
| Llama 3.1 8B Instruct | 67.8% | 2025-02-01 | independent | Source ↗ |
| Mistral Nemo | 67.8% | 2025-02-01 | independent | Source ↗ |
| Sarvam 2B (Sarvam-1) | 66.8% | 2025-02-01 | independent | Source ↗ |
| Phi-4-mini | 68.4% | 2025-02-01 | independent | Source ↗ |
| Amazon Nova Micro 1.0 | 64.2% | 2025-02-01 | independent | Source ↗ |
| Gemma 2 9B | 66.8% | 2025-02-01 | independent | Source ↗ |
| Gemma 2 2B | 56.4% | 2025-02-01 | independent | Source ↗ |
Human Baseline & Difficulty Horizon
Calibrated human reference points, specialist benchmarks, and ceiling thresholds.Measured Human Score68.5%Domain Expert Baseline
Baseline Protocol & Interpretation
Average score achieved by domain experts across monthly refreshed problem sets.
Metric & Scoring Methodology
Verification protocols, aggregation formulas, and specialized metric variants.Primary Metric:
Composite Score %Scoring Engine:composite
Dataset & Compute Cost
Evaluation volume, public availability, API pricing, and local hardware requirements.Total Dataset Size1,800Annotated evaluation items
Public Test SetPublicOpenly mirrored on repositories
Access GatingOpen AccessUnrestricted download
Evaluation LicenseOpen AccessDataset usage and redistribution terms
Frontier API Compute Cost:
$5 – $20 USD for full benchmark evaluation run on frontier APIs.
Recommended Local GPU Setup:
1x NVIDIA RTX 4090 (24GB) or A100 (40GB/80GB) via vLLM / SGLang
Official Dataset & Benchmark Files:Download / View Dataset Repository ↗
How to Run & Reproduce
Standardized evaluation protocols, CLI commands, and reproducible runner templates.Prompt Regimezero-shot
Reasoning Modedirect
Sampling Temp0
Pass@k Budgetk = 1
Tools & SandboxPure Text
Scoring Verifiercomposite
Option AEleutherAI LM-Evaluation-Harness (Open-Weight Models)
lm_eval --model hf --model_args pretrained=<model_path> --tasks livebench --batch_size autoOption BOpenCompass Evaluation Framework
opencompass --datasets livebench --models <model_config>Python APIDeterministic Inference Loop Snippet
# Standard API Evaluation Loop
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
)Standardized Reporting Requirement:
When publishing results for LiveBench (Continuous), always report the exact prompt template, few-shot exemplar ordering, sampling temperature (temperature=0), maximum reasoning budget tokens, and the precise timestamped model snapshot ID.
Contamination & Memorization Analysis
Audit of pretraining exposure risks, memorization vectors, and refresh policies.Overall Contamination Risk:LOW
Refresh Cadence:
Static fixed snapshot
Test Set Exposure:
Public on web / HuggingFace