Back
The Good Judgment Project - Cornell Info Blogs
webblogs.cornell.edu·blogs.cornell.edu/info2040/2015/11/23/the-good-judgment-p...
A student blog post summarizing the Good Judgment Project; relevant to AI safety discussions around forecasting AI risk and timelines, calibration of predictions, and the use of structured forecasting in policy and governance contexts.
Metadata
Importance: 28/100blog posteducational
Summary
This Cornell student blog post examines the Good Judgment Project (GJP), a forecasting research initiative that identified 'superforecasters' capable of significantly outperforming experts and intelligence analysts in predicting geopolitical events. It explores how structured probabilistic forecasting methods and diverse teams of skilled non-experts can produce surprisingly accurate predictions. The post highlights lessons about calibration, aggregation, and human judgment in forecasting.
Key Points
- •The Good Judgment Project, led by Philip Tetlock, found that aggregated forecasts from trained 'superforecasters' outperformed CIA analysts with access to classified information.
- •Superforecasters succeed through probabilistic thinking, frequent updating of beliefs, and breaking complex questions into tractable sub-problems.
- •The project demonstrates that forecasting skill is learnable and not confined to domain experts, challenging assumptions about expertise.
- •Calibration—matching stated confidence levels to actual accuracy—is a key metric distinguishing good forecasters from poor ones.
- •The work has implications for how organizations and policymakers can improve decision-making under uncertainty through structured forecasting methods.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Good Judgment (Forecasting) | Organization | 50.0 |
Cached Content Preview
HTTP 200Fetched Mar 15, 20264 KB
The Good Judgment Project and Geopolitical Forecasting : Networks Course blog for INFO 2040/CS 2850/Econ 2040/SOC 2090
Skip to main content
Search Cornell
Networks
Course blog for INFO 2040/CS 2850/Econ 2040/SOC 2090
The Good Judgment Project and Geopolitical Forecasting
Over the last two years, two researchers have conducted an experiment to explore whether or not there is a way to identify individuals with exceptional ability to predict geopolitical events. Together, Barbara Mellers and Michael C. Horowitz created the “Good Judgment Project” an online prediction market consisting of volunteers who had a Bachelor’s degree or higher. In all, the experiment yielded over 150,000 forecasts across 200 different questions and established a framework for what makes a “superforecaster.”
While it was primarily a psychological experiment aimed at establishing that aforementioned framework, the Good Judgment Project shows that the concept of prediction markets can be quite powerful in geopolitical contexts. The experiment’s findings show that one of the common pitfalls of supposed “experts” in geopolitical forecasting is the lack of concrete feedback. Predictions are made in accordance with doctrines and schools of thought rather than in any system that keeps score. Psychologically, this is the strength of prediction markets: they allow for instantaneous feedback and a greater sense of accountability. Rather than in the occasionally vague world of geopolitics, prediction markets create a numerical system that rewards participants for shifting forecasts because of past errors. This reward system that is predicated on shifting prediction styles directly confronts the concept of doctrinal thinking. More generally, the Good Judgment Project’s success indicates that prediction markets may very well be a part of geopolitical deliberations in the future.
Source:
https://www.washingtonpost.com/blogs/monkey-cage/wp/2015/01/29/does-anyone-make-accurate-geopolitical-predictions/
November 23, 2015 | category:
Uncategorized
Leave a Comment
-->
Comments
Leave a Reply
Name
(required)
Mail (will not be published)
(required)
Website
XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>
-->
Blog Admin
-->
Blogging Calendar
November 2015
M
T
W
T
F
S
S
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30
Google Search Archives
Archives
Select Month
December 2023 (77
... (truncated, 4 KB total)Resource ID:
05da93732c06a00b | Stable ID: MzJmMDc2MD