MMLU-Pro
KnowledgeA harder variant of MMLU with 10 answer choices (vs 4), chain-of-thought reasoning, and reduced sensitivity to prompt format. Designed to better discriminate among top models.
Models Tested
15
Best Score
84%
Median Score
76%
Scoring: accuracy
Introduced: 2024-06
Maintainer: TIGER Lab
Leaderboard15 models
| # | Model | Developer | Score |
|---|---|---|---|
| 🥇 | DeepSeek R1 | DeepSeek | 84% |
| 🥈 | Llama 4 Maverick | Meta AI (FAIR) | 80.5% |
| 🥉 | o1 | OpenAI | 80.3% |
| 4 | Grok-3 | xAI | 79.9% |
| 5 | Gemini 2.5 Pro | Google DeepMind | 79.1% |
| 6 | Claude 3.5 Sonnet | Anthropic | 78% |
| 7 | Gemini 2.0 Flash | Google DeepMind | 76.4% |
| 8 | Gemini 2.5 Flash | Google DeepMind | 76% |
| 9 | DeepSeek V3 | DeepSeek | 75.9% |
| 10 | GPT-4.1 | OpenAI | 74.3% |
| 11 | Llama 4 Scout | Meta AI (FAIR) | 74.3% |
| 12 | GPT-4o | OpenAI | 72.6% |
| 13 | Llama 3.3 | Meta AI (FAIR) | 68.9% |
| 14 | Gemini 1.5 Flash | Google DeepMind | 67.3% |
| 15 | GPT-4o mini | OpenAI | 63.2% |