Samotsvety Probabilities for Potential Precursors of Existential Risk
webSamotsvety forecasting group provides probability estimates for various catastrophic risks over the next 10 years, including AI-related risks, which is directly relevant to AI safety planning and prioritization of existential risk mitigation efforts.
Metadata
Importance: 62/100blog postanalysis
Summary
Seven Samotsvety forecasters aggregate probability estimates for various catastrophes causing over 1 million direct deaths in any single year within the next decade. The analysis covers risks ranging from war (23%) and natural pandemics (22%) to AI (10%) and biological weapons (2.2%), using geometric mean aggregation and best-model approaches. Results are contextualized for Sentinel's work on averting incipient catastrophes.
Key Points
- •War and natural pandemics are rated highest risk at ~23% and ~22% respectively for >1M deaths in a year over the next 10 years.
- •AI-related catastrophe is estimated at 10% probability, with a wide range of 1%-40% reflecting significant forecaster disagreement.
- •Biological weapons and unintentional biochem accidents are estimated at 2.2% and 2.4% respectively, lower than many might expect.
- •The forecasting methodology alternates between best-model estimates and geometric mean of odds depending on question type and data availability.
- •Results are intended to inform Sentinel's prioritization of catastrophe avoidance and mitigation strategies.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Samotsvety | Organization | 61.0 |
Cached Content Preview
HTTP 200Fetched Apr 28, 202628 KB
Samotsvety probabilities for potential precursors of existential risk
Seven Samotsvety forecasters give their probabilities for different possible catastrophes to lead to more than 1M direct deaths in any one year, in any of the next 10 years . This post presents results, starting with a table which presents the probabilities, continuing with a discussion of each of the items, and concluding with some thoughts on relevance for Sentinel ’s work averting or mitigating incipient catastrophes.
Some forecasters found some questions in particular much more interesting, so we alternate between presenting the probability of the best model when one such exists, and the geometric mean of the odds when it’s a risk that requires more subjective judgment and we have enough judgments to aggregate. Events that could fall into different buckets resolve each buckets. For instance, an AI-invented biological weapon used during WW3 would resolve the “AI”, “biological weapons” and “WW3” buckets.
Summary plot and table
Calamity
Probability > 1M deaths in a year of the next 10
Range 1
estimate
War
23%
18% to 40%
Forecaster aggregate
Natural pandemic
22%
10% to 30%
Forecaster aggregate
The unexpected
21%
15% to 30%
Forecaster aggregate
WW3 specifically
12%
5% to 40%
Forecaster aggregate
AI
10%
1% to 40%
Forecaster aggregate
Unintentional biochem
2.4%
0.8% to 10%
Forecaster aggregate
Biological weapons
2.2%
2% to 2.5%
Forecaster aggregate
Climate change
2.5%
1% to 5%
Forecaster aggregate
Volcanoes
1%
0.2% to 2%
Best model
US civil war
0.92%
0.05% to 3%
Forecaster aggregate
Financial crisis
0.59%
0.2% to 2%
Forecaster aggregate
Solar storm
0.2%
0.00066% to 0.71%
Best model
Simulation shutdown
0.13%
0.001% to 3%
Forecaster aggregate
Asteroid impact
0.02%
-
Best model
Alien invasion
0.00014%
0.00000025% to 0.1%
Forecaster aggregate
Vacuum decay, weird physics
<~1e-8
-
Best model
War
Type of estimate : Geometric mean of forecaster probabilities
Probabilities : 20%, 20%, 40%, 25%, 21.5%,
Range : 20% to 40%
Aggregate : 25%
Forecaster comments :
For context, Ukraine war is ~200k deaths over 2 years. WW1 and WW2 were 10M to 100M deaths
Baserate: WW1 and WW2 for last century are already massive wars, giving baserate of 2/100 years.
Specific spots: 15% for China/Taiwan leading to a conflict. But it doesn’t necessarily lead to 1M deaths. Say ~40% that it does
For reference, the Ukraine war hasn’t reached 1M/year.
I think I was previously higher, but have down based on vibes, and on accounting for 1M/year being a fairly high threshold.
ASI ~40% within 10 years, in that case maybe \~60% due to destabilization. Might be hard to classify instances of misaligned takeover. If
... (truncated, 28 KB total)Resource ID:
bb319ae43417f21d | Stable ID: sid_XdSu06pXoQ