Metaculus

Quick Assessment

Organization Details

Overview

Metaculus is a reputation-based online forecasting platform that has become the most influential source for AI timeline predictions. Founded in 2015 by physicists and data scientists from UC Santa Cruz and Yale, Metaculus aggregates predictions from thousands of forecasters on questions spanning technology, science, politics, and existential risks—with particular depth on AI capabilities and timelines.

The platform's core innovation is its aggregation algorithm, which combines individual forecasts into a single community estimate that consistently outperforms the simple median of user predictions when evaluated using Brier or Log scoring rules. The Community Prediction uses a recency-weighted median, while the now-deprecated "Metaculus Prediction" applied performance weighting and extremization to further improve accuracy. This aggregation enables Metaculus to extract signal from noise, producing calibrated probability estimates that inform research, policy, and public understanding.

Metaculus has documented a dramatic shift in AI timeline forecasts over recent years. The median estimate for when AGI would be developed fell from 50 years away in 2020 to approximately 5 years by late 2024. This shift—reflected in questions tracking "weak AGI," "general AI," and "transformative AI"—represents one of the most significant updates in the forecasting community's collective judgment about AI development trajectories.

The platform occupies a unique position in the AI safety ecosystem, providing quantitative probability estimates that inform decisions at organizations like Coefficient Giving, 80,000 Hours, and major AI labs. Its tournaments have also become important benchmarks for comparing human forecasters against AI forecasting systems, with human Pro Forecasters maintaining a statistically significant lead over AI bots through 2025.

History and Founding

Origins (2014-2015)

Metaculus emerged from the intersection of cosmological research and technology forecasting. Co-founder Anthony Aguirre, a theoretical cosmologist and Faggin Presidential Chair for the Physics of Information at UC Santa Cruz, had previously co-founded the Foundational Questions Institute (FQXi) in 2006 with Max TegmarkPersonMax TegmarkComprehensive biographical profile of Max Tegmark covering his transition from cosmology to AI safety advocacy, his role founding the Future of Life Institute, and his controversial Mathematical Un...Quality: 63/100 to support unconventional physics research. In 2014, Aguirre and Tegmark co-founded the Future of Life Institute (FLI), an organization focused on the implications of transformational technology, particularly artificial intelligence.

Metaculus was conceived alongside FLI as a complementary tool. According to Aguirre, the forecasting platform was designed to "build an ability to make predictions and identify people who are really good at making predictions and modeling the world." The platform started partly to "be a thing that would be of service to the Future of Life InstituteOrganizationFuture of Life InstituteComprehensive profile of FLI documenting $25M+ in grants distributed (2015: $7M to 37 projects, 2021: $25M program), major public campaigns (Asilomar Principles with 5,700+ signatories, 2023 Pause ...Quality: 46/100, but also everybody else who's thinking about the future" (AXRP Episode 38.7).

Aguirre partnered with Greg Laughlin, a Yale astrophysicist with expertise in computational methods and orbital dynamics, and Max Wainwright, a data scientist who had been a postdoctoral researcher for both Laughlin and Aguirre. The trio launched Metaculus in November 2015, initially focusing on science and technology predictions where their academic expertise provided domain knowledge.

Platform Development (2016-2019)

The early platform focused on building a community of technically-minded forecasters and developing the mathematical infrastructure for prediction aggregation. In June 2017, Metaculus introduced the Metaculus Prediction—a sophisticated aggregation system that weighted forecasts based on past performance and applied extremization to compensate for systematic human cognitive biases. This innovation helped distinguish Metaculus from simple prediction aggregators.

The platform gradually expanded from its initial science and technology focus to include questions on geopolitics, economics, and global risks. This expansion positioned Metaculus to become a central resource for the effective altruism community, which was increasingly interested in quantitative forecasts for cause prioritization and career decisions.

Growth and Institutionalization (2020-2022)

The COVID-19 pandemic marked a turning point for Metaculus's public profile. In January 2020, Metaculus introduced the Bentham Prize, awarding bi-weekly monetary prizes of USD 300, USD 200, and USD 100 to the most valuable user contributions. In February 2020, they launched the Li Wenliang Prize, named after the Chinese doctor who warned about COVID-19, offering monetary prizes for questions, forecasts, and analyses related to the outbreak.

The pandemic demonstrated the platform's ability to rapidly aggregate expert judgment on developing situations, attracting significant attention from researchers and policymakers. By 2022, Metaculus reached 1,000,000 individual predictions and restructured as a public-benefit corporation, signaling a commitment to forecasting as a public good rather than a purely commercial venture.

Scaling with Major Funding (2022-Present)

Coefficient Giving's USD 5.5 million grant in 2022 transformed Metaculus's capacity, enabling significant hiring and platform development. The funding supported high-impact forecasting programs on AI, biosecurity, climate change, nuclear security, and other topics of concern to the longtermist community. A follow-up USD 3 million grant in 2023 further expanded capabilities.

In April 2024, Deger Turan became CEO, bringing experience from his role heading the AI Objectives Institute. The previous CEO transitioned to Special Advisor while remaining on the board. Under Turan's leadership, Metaculus launched major AI forecasting initiatives including the AI Forecasting Benchmark TournamentProjectAI Forecasting Benchmark TournamentQuarterly competition (Q2 2025: 348 questions, 54 bot-makers, $30K prizes) comparing human Pro Forecasters against AI bots, with statistical testing showing humans maintain significant lead (p=0.00...Quality: 41/100, which benchmarks AI forecasting systems against human Pro Forecasters.

In 2024, Metaculus rewrote their website code and released it under the BSD-2-Clause License, making their platform open source. The AI Forecasting Benchmark Series continued into 2025, with Q1 results prompting a renewal announcement in July 2025 for an expanded year-long iteration backed by USD 175,000 in prizes.

Team and Leadership

The organization has grown to approximately 28 employees across three continents (North America, Europe, and Asia), reflecting its global forecaster community (Tracxn).

Key AGI Timeline Forecasts

flowchart LR
  2020[2020<br/>50 years to AGI] --> 2022[2022<br/>30 years to AGI]
  2022 --> 2024[Late 2024<br/>~7 years to AGI]
  2024 --> 2025[2025<br/>~5 years to AGI]

  style 2020 fill:#ffcccc
  style 2022 fill:#ffddcc
  style 2024 fill:#ffffcc
  style 2025 fill:#ccffcc

Current AGI Probability Estimates (as of late 2024)

Metaculus AGI Definition

Metaculus uses a multi-criteria definition requiring systems to:

This definition is more stringent than industry definitions (e.g., OpenAI's "economically valuable work"), leading to somewhat later timeline estimates compared to lab predictions.

Platform Statistics

Accuracy and Calibration

Metaculus Prediction vs. Community Median

The Metaculus Prediction aggregation algorithm provides measurable improvements:

Calibration Analysis

One EA Forum analysis found Metaculus was "poorly calibrated on resolved questions with a greater than 1 year time horizon," suggesting caution when interpreting long-range AI forecasts.

Predictor Quantity Effects

Research on Metaculus data shows diminishing returns to additional forecasters:

Aggregation Methodology

Metaculus employs sophisticated aggregation techniques to combine individual forecasts into community estimates. Understanding these methods is essential for interpreting the platform's predictions.

Community Prediction: Recency-Weighted Median

The Community Prediction uses a recency-weighted median approach:

For different question types (Metaculus FAQ):

• Binary Questions: Weighted median of individual probabilities
• Multiple Choice: Weighted median, renormalized to sum to 1
• Numeric/Date Questions: Weighted average of individual distributions

Metaculus Prediction: Performance Weighting + Extremization

The "Metaculus Prediction" (deprecated since November 2024) employed a more sophisticated approach (Metaculus Notebooks):

How Extremization Works (EA Forum):

Extremizing adjusts aggregated forecasts toward extreme probabilities. The rationale: if several independent forecasters conclude something is 90% likely, their agreement provides additional evidence beyond any individual's estimate. Research on geopolitical forecasting found optimal extremizing factors between 1.161 and 3.921.

Scoring System

Metaculus uses a logarithmic scoring rule as the foundation for all scores (Metaculus Scoring Primer):

Key properties of the log score:

• Proper scoring rule: The only way to optimize average score is to predict sincere beliefs
• Punitive on extreme errors: Going from 99% to 99.9% yields only +0.009 if correct, but -2.3 if wrong
• Time-averaged: Points are averaged across question lifetime to encourage ongoing updates

The November 2023 scoring update replaced the legacy Points system with Baseline and Peer scores, making performance comparison fairer for forecasters with different time constraints.

flowchart TD
  subgraph INPUT["Individual Forecasts"]
      F1[Forecaster 1<br/>75%]
      F2[Forecaster 2<br/>80%]
      F3[Forecaster 3<br/>70%]
      FN[Forecaster N<br/>...]
  end

  subgraph WEIGHT["Recency Weighting"]
      RW[Older forecasts<br/>weight = 1]
      RW2[Newer forecasts<br/>weight = n]
  end

  subgraph AGG["Aggregation"]
      MED[Weighted Median]
      CP[Community Prediction]
  end

  subgraph LEGACY["Historical - Deprecated Nov 2024"]
      PW[Performance<br/>Weighting]
      EXT[Extremization]
      MP[Metaculus<br/>Prediction]
  end

  INPUT --> WEIGHT
  WEIGHT --> MED
  MED --> CP
  MED --> PW
  PW --> EXT
  EXT --> MP

  style CP fill:#90ee90
  style MP fill:#ffcccc
  style LEGACY fill:#f5f5f5

Key AI Questions

Primary Timeline Questions

Company Attribution Forecasts

Metaculus forecasters have also estimated which organization is most likely to achieve AGI first (Forecastingaifutures.substack.com):

AI-Specific Tournaments

Tournaments and Competitions

AI Forecasting Benchmark Tournament

The AI Forecasting Benchmark Tournament represents Metaculus's flagship initiative for comparing human and AI forecasting capabilities. Launched in 2024, the tournament runs in two series: a primary 4-month seasonal tournament and a bi-weekly fast-paced MiniBench. Participants can compete using API credits provided by OpenAI and Anthropic.

Key findings from the Q2 2025 tournament (EA Forum):

• Students and hobbyists performed well—the top 3 bot-makers (excluding Metaculus's in-house bots) were hobbyists or students
• Aggregation had a significant positive effect: taking the median or mean of multiple forecasts rather than single LLM forecasts improved scores
• Among baseline bots in Q2 2025, OpenAI's o3 led performance rankings
• The average Peer score for the Metaculus Community Prediction is 12.9, ranking in the top 10 on the global leaderboard over every 2-year period since 2016

Bridgewater x Metaculus Forecasting Competition

Metaculus has partnered with Bridgewater Associates for three consecutive years on forecasting competitions designed to identify talented forecasters for potential recruitment. The competition emphasizes Bridgewater's "idea meritocracy" culture.

The January 2026 competition features:

• 50 forecasting questions on real-world events
• Separate leaderboards for open competition and undergraduates
• Top 50 forecasters in each track eligible for prizes
• Potential employment opportunities at Bridgewater for top performers

Good Judgment Inc. Collaboration

Metaculus and Good Judgment Inc. announced their first formal collaboration on the Our World in Data (OWID) project, comparing methodologies across the two largest human judgment forecasting communities globally.

According to Warren Hatch, Good Judgment's CEO: "We're excited to be partnering with Metaculus to combine our approaches to apply probabilistic thinking to an uncertain future."

Vox Future Perfect Collaboration

In January 2025, Metaculus partnered with Vox's Future Perfect team to host forecasts on political, economic, and technological questions for 2025, featuring:

• Public participation alongside the Future Perfect team's published predictions
• USD 2,500 prize pool to reward accurate contributions

Tournament Summary

Comparison with Other Platforms

Platform Characteristics

Accuracy Comparison Studies

Research comparing forecasting platform accuracy has produced nuanced findings (Manifund research, Metaculus notebooks):

AGI Timeline Comparison (2024-2025)

The AGI Timelines Dashboard aggregates data from Metaculus, Manifold, Kalshi, and other sources, producing a combined forecast of AGI arriving in 2031 (80% CI: 2027-2045) as of January 2026.

Why Estimates Differ

Funding History

Major Grants

Total Funding: USD 8.8M+ confirmed

Coefficient Giving has been Metaculus's primary funder, providing support under its Longtermism program, which focuses on work that "raises the probability of a very long-lasting, positive future" (Metaculus announcement). The USD 5.5M 2022 grant was described as enabling Metaculus to "scale as an organization pursuing its mission to build epistemic infrastructure for navigating complex global challenges."

Funding Focus Areas

Coefficient Giving's grants to Metaculus support high-impact forecasting programs in:

Partnerships and Collaborations

Industry and Research Partners

Academic and Institutional Connections

The organization's founders maintain deep connections to academic research and global risk institutions:

Strengths and Limitations

Strengths

Limitations

AI-Specific Track Record Analysis

The Metaculus AI Progress Tournament analysis found:

This pattern suggests: "AI progresses surprisingly rapidly on well-defined benchmarks but the attention it receives and its 'real world' impact fail to keep up."

Relevance to AI Safety

Metaculus plays several important roles in the AI safety ecosystem:

Decision Support

Epistemic Infrastructure

Metaculus provides quantitative probability estimates where previously only qualitative assessments existed. The dramatic shift from 50-year to 5-year AGI timelines between 2020 and 2024 represents one of the most significant and well-documented updates in collective expert judgment about AI development, providing valuable signal for resource allocation and urgency calibration.

AI Capability Benchmarking

The AI Forecasting Benchmark Tournament provides empirical data on the state of AI forecasting capabilities:

These benchmarks help calibrate expectations about AI capability trajectories in reasoning-intensive domains.

External Links

• Metaculus Homepage
• AI Questions Category
• AI Forecasting Benchmark Tournament
• Track Record Page
• Metaculus FAQ
• AGI Timelines Dashboard (aggregates Metaculus + others)
• Wikipedia

Sources

1. Metaculus - Wikipedia - Organization history and overview
2. Anthony Aguirre - Wikipedia - Founder background
3. Anthony Aguirre - Future of Life Institute - FLI connection
4. Metaculus: a prediction website with an eye on science and technology - Yale News - 2016 launch coverage
5. Announcing Deger Turan as the new CEO of Metaculus - Leadership transition
6. Metaculus Awarded USD 5.5M Grant to Advance Forecasting as a Public Good - 2022 funding announcement
7. Metaculus - Platform Development | Coefficient Giving - 2022 grant details
8. Metaculus - Platform Development (2023) | Coefficient Giving - 2023 grant details
9. Good Judgment Inc and Metaculus Launch First Collaboration - OWID project announcement
10. Bridgewater x Metaculus 2026 Competition - Bridgewater partnership
11. Q2 AI Benchmark Results: Pros Maintain Clear Lead - EA Forum - AI benchmarking results
12. How does forecast quantity impact forecast quality on Metaculus? - EA Forum - Calibration analysis
13. Takeaways from the Metaculus AI Progress Tournament - EA Forum - AI tournament analysis
14. Takeaways from the Metaculus AI Progress Tournament | Coefficient Giving - Coefficient Giving analysis
15. A Primer on the Metaculus Scoring Rule - Scoring methodology
16. Exploring Metaculus's AI Track Record - EA Forum - AI accuracy analysis
17. Predictive Performance on Metaculus vs. Manifold Markets - Platform comparison
18. Shrinking AGI timelines: a review of expert forecasts | 80,000 Hours - Timeline analysis
19. Forecasting AGI: Insights from Prediction Markets and Metaculus - AGI forecast aggregation
20. Data on forecasting accuracy across different time horizons - EA Forum - Long-horizon analysis
21. AXRP Episode 38.7 - Anthony Aguirre on the Future of Life Institute - Founder interview
22. Metaculus Year in Review: 2022 - 1M prediction milestone
23. Metaculus Company Profile - Tracxn - Organization details
24. Principled extremizing of aggregated forecasts - EA Forum - Extremization methodology
25. What can we learn from scoring different election forecasts? - First Sigma - 2022 election comparison