ForecastBench: Dynamic LLM Forecasting Benchmark
webForecastBench is a dynamic benchmark measuring LLM forecasting accuracy against human baselines, relevant to AI safety as forecasting ability serves as a proxy for general intelligence and helps track AI capability progress toward and beyond human-level performance.
Metadata
Importance: 62/100tool pagetool
Summary
ForecastBench is a contamination-free benchmark that evaluates LLM forecasting accuracy against human comparison groups, including superforecasters. It maintains both a baseline leaderboard (no tools) and a tournament leaderboard (with scaffolding/tools), and projects when LLMs will reach superforecaster-level performance.
Key Points
- •Dynamic, contamination-free benchmark preventing LLMs from training on benchmark questions, ensuring valid capability measurement.
- •Compares LLM forecasting performance against human baselines including superforecasters as a proxy for general intelligence.
- •Dual leaderboards: baseline (raw model performance) and tournament (with tool use, fine-tuning, ensembling).
- •Tracks historical progress in LLM forecasting capabilities and projects date of LLM-superforecaster parity.
- •Open to public submissions, enabling broad participation in capability evaluation.
Cited by 2 pages
| Page | Type | Quality |
|---|---|---|
| Forecasting Research Institute (FRI) | Organization | 55.0 |
| ForecastBench | Project | 53.0 |
1 FactBase fact citing this source
| Entity | Property | Value | As Of |
|---|---|---|---|
| ForecastBench | Founded Date | Sep 2024 | — |
Cached Content Preview
HTTP 200Fetched Nov 3, 20262 KB
Tournament leaderboard Tracks frontier accuracy by allowing tool use to improve LLM performance. Models can be scaffolded, fine-tuned, ensembled, and so on. Open to public submissions . Tournament leaderboard Show 102550100All entriesSearch: RankOrgModelOverall 3fire hedgehog68.94ceramic-kettle68.74ensemble 168.74ensemble 368.77blue croc68.59Superforecaster median forecast68.410captain-jack68.310Cassi-2026-05-1068.313rice-demon68.214big green leaf68.1 Showing 1 to 10 of 439 entrieslast updated 2026-11-02PreviousNext Show 102550100All entriesSearch: RankOrgModelOverall 1Superforecaster median forecast67.32Public median forecast62.23kimi-k3-max-128k61.33o3-2025-04-16-scratchpad61.35claude-3-7-sonnet-20250219-scratchpad60.55minimax-m3-adaptive-thinking-1200060.57claude-opus-4-1-2025080560.47o4-mini-2025-04-1660.47o4-mini-2025-04-16-scratchpad60.47gpt-4.5-preview-2025-02-2760.4 Showing 1 to 10 of 130 entrieslast updated 2026-11-02PreviousNext Baseline leaderboard Tracks base model LLM forecasting performance without additional tools, comparing against human baselines and showing consistent progress in capabilities since models were first tested. Baseline leaderboard Projected LLM-superforecaster parity Explore how LLM forecasting accuracy evolves on ForecastBench. A linear trend projects the date when LLMs reach superforecaster-level performance. Explore chart PublicSuperforecasterGPT-4Claude 3 OpusGPT-4 TurboClaude 3.5 Sonnetgrok beta†Claude 3.7 Sonnetgrok 4 0709†gpt 5 2025Cassi ensemble 2green plantGrok 4.20Cassi 2026 05fire hedgehogmodel 4Jul 2023Oct 2023Jan 2024Apr 2024Jul 2024Oct 2024Jan 2025Apr 2025Jul 2025Oct 2025Jan 2026Apr 2026Jul 2026Oct 202640455055606570758085Model release dateBrier Index (%)Linear TrendSOTA ModelNon-SOTA ModelLLM-superforecaster parity achieved: June 07, 2026
Resource ID:
kb-c808dd961e2e3c1d | Stable ID: sid_WNgUtLr8jP