Skip to content
Longterm Wiki

Samotsvety Probabilities for Potential Precursors of Existential Risk

web

Samotsvety forecasting group provides probability estimates for various catastrophic risks over the next 10 years, including AI-related risks, which is directly relevant to AI safety planning and prioritization of existential risk mitigation efforts.

Metadata

Importance: 62/100blog postanalysis

Summary

Seven Samotsvety forecasters aggregate probability estimates for various catastrophes causing over 1 million direct deaths in any single year within the next decade. The analysis covers risks ranging from war (23%) and natural pandemics (22%) to AI (10%) and biological weapons (2.2%), using geometric mean aggregation and best-model approaches. Results are contextualized for Sentinel's work on averting incipient catastrophes.

Key Points

  • War and natural pandemics are rated highest risk at ~23% and ~22% respectively for >1M deaths in a year over the next 10 years.
  • AI-related catastrophe is estimated at 10% probability, with a wide range of 1%-40% reflecting significant forecaster disagreement.
  • Biological weapons and unintentional biochem accidents are estimated at 2.2% and 2.4% respectively, lower than many might expect.
  • The forecasting methodology alternates between best-model estimates and geometric mean of odds depending on question type and data availability.
  • Results are intended to inform Sentinel's prioritization of catastrophe avoidance and mitigation strategies.

Cited by 1 page

PageTypeQuality
SamotsvetyOrganization61.0

Cached Content Preview

HTTP 200Fetched Apr 28, 202628 KB
Samotsvety probabilities for potential precursors of existential risk

 Seven Samotsvety forecasters give their probabilities for different possible catastrophes to lead to more than 1M direct deaths in any one year, in any of the next 10 years . This post presents results, starting with a table which presents the probabilities, continuing with a discussion of each of the items, and concluding with some thoughts on relevance for Sentinel ’s work averting or mitigating incipient catastrophes.

 Some forecasters found some questions in particular much more interesting, so we alternate between presenting the probability of the best model when one such exists, and the geometric mean of the odds when it’s a risk that requires more subjective judgment and we have enough judgments to aggregate. Events that could fall into different buckets resolve each buckets. For instance, an AI-invented biological weapon used during WW3 would resolve the “AI”, “biological weapons” and “WW3” buckets.

 Summary plot and table

 

 
 
 
 Calamity 
 Probability > 1M deaths in a year of the next 10 
 Range 1 
 estimate 
 
 
 
 
 War 
 23% 
 18% to 40% 
 Forecaster aggregate 
 
 
 Natural pandemic 
 22% 
 10% to 30% 
 Forecaster aggregate 
 
 
 The unexpected 
 21% 
 15% to 30% 
 Forecaster aggregate 
 
 
 WW3 specifically 
 12% 
 5% to 40% 
 Forecaster aggregate 
 
 
 AI 
 10% 
 1% to 40% 
 Forecaster aggregate 
 
 
 Unintentional biochem 
 2.4% 
 0.8% to 10% 
 Forecaster aggregate 
 
 
 Biological weapons 
 2.2% 
 2% to 2.5% 
 Forecaster aggregate 
 
 
 Climate change 
 2.5% 
 1% to 5% 
 Forecaster aggregate 
 
 
 Volcanoes 
 1% 
 0.2% to 2% 
 Best model 
 
 
 US civil war 
 0.92% 
 0.05% to 3% 
 Forecaster aggregate 
 
 
 Financial crisis 
 0.59% 
 0.2% to 2% 
 Forecaster aggregate 
 
 
 Solar storm 
 0.2% 
 0.00066% to 0.71% 
 Best model 
 
 
 Simulation shutdown 
 0.13% 
 0.001% to 3% 
 Forecaster aggregate 
 
 
 Asteroid impact 
 0.02% 
 - 
 Best model 
 
 
 Alien invasion 
 0.00014% 
 0.00000025% to 0.1% 
 Forecaster aggregate 
 
 
 Vacuum decay, weird physics 
 <~1e-8 
 - 
 Best model 
 
 
 

 War

 Type of estimate : Geometric mean of forecaster probabilities

 Probabilities : 20%, 20%, 40%, 25%, 21.5%,

 Range : 20% to 40%

 Aggregate : 25%

 Forecaster comments :

 For context, Ukraine war is ~200k deaths over 2 years. WW1 and WW2 were 10M to 100M deaths

 

 

 Baserate: WW1 and WW2 for last century are already massive wars, giving baserate of 2/100 years.
Specific spots: 15% for China/Taiwan leading to a conflict. But it doesn&rsquo;t necessarily lead to 1M deaths. Say ~40% that it does

For reference, the Ukraine war hasn&rsquo;t reached 1M/year.
I think I was previously higher, but have down based on vibes, and on accounting for 1M/year being a fairly high threshold.

 

 

 ASI ~40% within 10 years, in that case maybe \~60% due to destabilization. Might be hard to classify instances of misaligned takeover. If 

... (truncated, 28 KB total)
Resource ID: bb319ae43417f21d | Stable ID: sid_XdSu06pXoQ