Skip to content
Longterm Wiki has not been actively maintained since May 2026. Some data is still updated occasionally, but please don't rely on this site being current or accurate.
Longterm Wiki

ForecastBench: Dynamic LLM Forecasting Benchmark

web
forecastbench.org·forecastbench.org

ForecastBench is a dynamic benchmark measuring LLM forecasting accuracy against human baselines, relevant to AI safety as forecasting ability serves as a proxy for general intelligence and helps track AI capability progress toward and beyond human-level performance.

Metadata

Importance: 62/100tool pagetool

Summary

ForecastBench is a contamination-free benchmark that evaluates LLM forecasting accuracy against human comparison groups, including superforecasters. It maintains both a baseline leaderboard (no tools) and a tournament leaderboard (with scaffolding/tools), and projects when LLMs will reach superforecaster-level performance.

Key Points

  • •Dynamic, contamination-free benchmark preventing LLMs from training on benchmark questions, ensuring valid capability measurement.
  • •Compares LLM forecasting performance against human baselines including superforecasters as a proxy for general intelligence.
  • •Dual leaderboards: baseline (raw model performance) and tournament (with tool use, fine-tuning, ensembling).
  • •Tracks historical progress in LLM forecasting capabilities and projects date of LLM-superforecaster parity.
  • •Open to public submissions, enabling broad participation in capability evaluation.

Cited by 2 pages

PageTypeQuality
Forecasting Research Institute (FRI)Organization55.0
ForecastBenchProject53.0

1 FactBase fact citing this source

EntityPropertyValueAs Of
ForecastBenchFounded DateSep 2024—

Cached Content Preview

HTTP 200Fetched Nov 3, 20262 KB
Tournament leaderboard
 Tracks frontier accuracy by allowing tool use to improve LLM performance. Models can be scaffolded, fine-tuned, ensembled, and so on. Open to public submissions .
 Tournament leaderboard
 
 
 
 
 
 Show 102550100All entriesSearch:
 
 RankOrgModelOverall
 3fire hedgehog68.94ceramic-kettle68.74ensemble 168.74ensemble 368.77blue croc68.59Superforecaster median forecast68.410captain-jack68.310Cassi-2026-05-1068.313rice-demon68.214big green leaf68.1
 Showing 1 to 10 of 439 entrieslast updated 2026-11-02PreviousNext
 
 
 
 

 
 

 
 

 
 
 
 
 Show 102550100All entriesSearch:
 
 RankOrgModelOverall
 1Superforecaster median forecast67.32Public median forecast62.23kimi-k3-max-128k61.33o3-2025-04-16-scratchpad61.35claude-3-7-sonnet-20250219-scratchpad60.55minimax-m3-adaptive-thinking-1200060.57claude-opus-4-1-2025080560.47o4-mini-2025-04-1660.47o4-mini-2025-04-16-scratchpad60.47gpt-4.5-preview-2025-02-2760.4
 Showing 1 to 10 of 130 entrieslast updated 2026-11-02PreviousNext
 
 
 
 
 
 Baseline leaderboard
 Tracks base model LLM forecasting performance without additional tools, comparing against human baselines and showing consistent progress in capabilities since models were first tested.
 Baseline leaderboard
 
 
 

 
 

 
 

 
 
 Projected LLM-superforecaster parity
 
 Explore how LLM forecasting accuracy evolves on ForecastBench. A linear trend projects the date when LLMs reach superforecaster-level performance.
 Explore chart
 
 
 
 PublicSuperforecasterGPT-4Claude 3 OpusGPT-4 TurboClaude 3.5 Sonnetgrok beta†Claude 3.7 Sonnetgrok 4 0709†gpt 5 2025Cassi ensemble 2green plantGrok 4.20Cassi 2026 05fire hedgehogmodel 4Jul 2023Oct 2023Jan 2024Apr 2024Jul 2024Oct 2024Jan 2025Apr 2025Jul 2025Oct 2025Jan 2026Apr 2026Jul 2026Oct 202640455055606570758085Model release dateBrier Index (%)Linear TrendSOTA ModelNon-SOTA ModelLLM-superforecaster parity achieved: June 07, 2026
Resource ID: kb-c808dd961e2e3c1d | Stable ID: sid_WNgUtLr8jP