Back
The Good Judgment Project - Wikipedia
referenceCredibility Rating
3/5
Good(3)Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: Wikipedia
Relevant to AI safety as forecasting methodology and superforecaster concepts are widely used in the EA/AI safety community for assessing AI risk timelines and evaluating judgment under uncertainty.
Metadata
Importance: 45/100wiki pagereference
Summary
The Good Judgment Project was a large-scale forecasting research initiative that competed in IARPA's Aggregative Contingent Estimation (ACE) program, demonstrating that structured training and aggregation methods can significantly improve geopolitical forecasting accuracy. It identified 'superforecasters'—individuals with exceptional predictive ability—and showed that teams using calibration techniques consistently outperformed intelligence analysts with access to classified information. The project laid groundwork for understanding how to improve human judgment under uncertainty.
Key Points
- •Participated in IARPA's ACE program, consistently outperforming other forecasting methods including prediction markets and expert analysts.
- •Identified 'superforecasters'—a small subset of forecasters with consistently superior calibration and accuracy across diverse questions.
- •Demonstrated that training in probabilistic thinking and aggregating diverse forecasts substantially improves prediction quality.
- •Results influenced the development of structured forecasting methods relevant to AI risk and long-term planning.
- •Philip Tetlock and Barbara Mellers led the project, building on Tetlock's earlier work on expert political judgment.
Cited by 2 pages
| Page | Type | Quality |
|---|---|---|
| Forecasting Research Institute (FRI) | Organization | 55.0 |
| Good Judgment (Forecasting) | Organization | 50.0 |
Cached Content Preview
HTTP 200Fetched Mar 15, 202631 KB
# The Good Judgment Project
The Good Judgment Project
Crowd-sourced event prediction
**The Good Judgment Project** ( **GJP**) is an organization dedicated to "harnessing the wisdom of the crowd to forecast world events". It was co-created by [Philip E. Tetlock](https://en.wikipedia.org/wiki/Philip_E._Tetlock "Philip E. Tetlock") (author of _**Superforecasting: The Art and Science of Prediction**_ and _[Expert Political Judgment](https://en.wikipedia.org/wiki/Expert_Political_Judgment "Expert Political Judgment")_), decision scientist [Barbara Mellers](https://en.wikipedia.org/wiki/Barbara_Mellers "Barbara Mellers"), and [Don Moore](https://en.wikipedia.org/wiki/Don_A._Moore_(academic) "Don A. Moore (academic)"), all professors at the [University of Pennsylvania](https://en.wikipedia.org/wiki/University_of_Pennsylvania "University of Pennsylvania").[\[1\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-main-1)[\[2\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-2)[\[3\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-nytimes-3)
The project began as a participant in the [Aggregative Contingent Estimation](https://en.wikipedia.org/wiki/Aggregative_Contingent_Estimation "Aggregative Contingent Estimation") (ACE) program of the [Intelligence Advanced Research Projects Activity](https://en.wikipedia.org/wiki/Intelligence_Advanced_Research_Projects_Activity "Intelligence Advanced Research Projects Activity") (IARPA).[\[4\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-project-4)[\[5\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-washpo-5) It then extended its [crowd wisdom](https://en.wikipedia.org/wiki/Wisdom_of_the_crowd "Wisdom of the crowd") to commercial activities, recruiting forecasters and aggregating the predictions of the most historically accurate among them to [forecast future events](https://en.wikipedia.org/wiki/Forecasting "Forecasting").[\[6\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-6)[\[7\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-7) Predictions are [scored](https://en.wikipedia.org/wiki/Scoring_rule "Scoring rule") using [Brier scores](https://en.wikipedia.org/wiki/Brier_score "Brier score").[\[8\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-predictive-heuristics-icews-8) The top forecasters in GJP are "reportedly 30% better than intelligence officers with access to actual classified information."[\[9\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-:0-9)
## History
The Good Judgment Project began in July 2011 in collaboration with the [Aggregative Contingent Estimation (ACE) Program](https://en.wikipedia.org/wiki/Aggregative_Contingent_Estimation_(ACE)_Program "Aggregative Contingent Estimation (ACE) Program") at IARPA (IARPA-ACE).[\[10\]](https://en.wikipedia.org/wiki/The_Good_Judgment_Project#cite_note-10) The first contest began in
... (truncated, 31 KB total)Resource ID:
005e69824dedfd4b | Stable ID: MWNmMDJmMm