Page StatusContent
Edited 7 weeks ago2.9k words2 backlinks
66
QualityGoodQuality: 66/100Human-assigned rating of overall page quality, considering depth, accuracy, and completeness.82
ImportanceHighImportance: 82/100How central this topic is to AI safety. Higher scores mean greater relevance to understanding or mitigating AI risk.10
Structure10/15Structure: 10/15Automated score based on measurable content features.Word count2/2Tables3/3Diagrams0/2Internal links2/2Citations0/3Prose ratio2/2Overview section1/126TablesData tables in the page0DiagramsCharts and visual diagrams68Internal LinksLinks to other wiki pages0FootnotesFootnote citations [^N] with sources0External LinksMarkdown links to outside URLs%16%Bullet RatioPercentage of content in bullet lists
Updated quarterlyDue in 6 weeks
Summary
Comprehensive framework mapping AI risk activation windows with specific probability assessments: current risks already active (disinformation 95%+, spear phishing active), near-term critical window 2025-2027 (bioweapons 50% by 2027, cyberweapons 75%), long-term existential risks 2030-2050+ (ASI misalignment 15% by 2030). Recommends $3-5B annual investment in Tier 1 interventions with specific allocations: $200-400M bioweapons screening, $300-600M interpretability, $500M-1B cyber-defense.
TODOs2
Complete 'Conceptual Framework' section
Complete 'Quantitative Analysis' section (8 placeholders)
Risk Activation Timeline Model
Model
AI Risk Activation Timeline Model
Comprehensive framework mapping AI risk activation windows with specific probability assessments: current risks already active (disinformation 95%+, spear phishing active), near-term critical window 2025-2027 (bioweapons 50% by 2027, cyberweapons 75%), long-term existential risks 2030-2050+ (ASI misalignment 15% by 2030). Recommends $3-5B annual investment in Tier 1 interventions with specific allocations: $200-400M bioweapons screening, $300-600M interpretability, $500M-1B cyber-defense.
Model TypeTimeline Projection
ScopeCross-cutting (all risk categories)
Key InsightRisks activate at different times based on capability thresholds
Related
Models
AI Capability Threshold ModelModelAI Capability Threshold ModelComprehensive framework mapping AI capabilities across 5 dimensions to specific risk thresholds, finding authentication collapse/mass persuasion risks at 70-85% likelihood by 2027, bioweapons devel...Quality: 72/100AI Risk Warning Signs ModelModelAI Risk Warning Signs ModelSystematic framework for detecting AI risks through 32 warning signs across 5 categories, finding critical indicators are 18-48 months from thresholds with 45-90% detection probability, but only 30...Quality: 70/100AI-Bioweapons Timeline ModelModelAI-Bioweapons Timeline ModelTimeline model projects AI-bioweapons capabilities crossing four thresholds: knowledge democratization already partially crossed (fully by 2025-2027), synthesis assistance 2027-2032 (median 2029), ...Quality: 58/100
2.9k words · 2 backlinks
Model
AI Risk Activation Timeline Model
Comprehensive framework mapping AI risk activation windows with specific probability assessments: current risks already active (disinformation 95%+, spear phishing active), near-term critical window 2025-2027 (bioweapons 50% by 2027, cyberweapons 75%), long-term existential risks 2030-2050+ (ASI misalignment 15% by 2030). Recommends $3-5B annual investment in Tier 1 interventions with specific allocations: $200-400M bioweapons screening, $300-600M interpretability, $500M-1B cyber-defense.
Model TypeTimeline Projection
ScopeCross-cutting (all risk categories)
Key InsightRisks activate at different times based on capability thresholds
Related
Models
AI Capability Threshold ModelModelAI Capability Threshold ModelComprehensive framework mapping AI capabilities across 5 dimensions to specific risk thresholds, finding authentication collapse/mass persuasion risks at 70-85% likelihood by 2027, bioweapons devel...Quality: 72/100AI Risk Warning Signs ModelModelAI Risk Warning Signs ModelSystematic framework for detecting AI risks through 32 warning signs across 5 categories, finding critical indicators are 18-48 months from thresholds with 45-90% detection probability, but only 30...Quality: 70/100AI-Bioweapons Timeline ModelModelAI-Bioweapons Timeline ModelTimeline model projects AI-bioweapons capabilities crossing four thresholds: knowledge democratization already partially crossed (fully by 2025-2027), synthesis assistance 2027-2032 (median 2029), ...Quality: 58/100
2.9k words · 2 backlinks
Overview
Different AI risks don't all "turn on" at the same time - they activate based on capability thresholds, deployment contexts, and barrier erosion. This model systematically maps when various AI risks become critical, enabling strategic resource allocation and intervention timing.
The model reveals three critical insights: many serious risks are already active with current systems, the next 2-3 years represent a critical activation window for multiple high-impact risks, and long-term existential risks require foundational research investment now despite uncertain timelines.
Understanding activation timing enables prioritizing immediate interventions for active risks, preparing defenses for near-term thresholds, and building foundational capacity for long-term challenges before crisis mode sets in.
Risk Assessment Overview
| Risk Category | Timeline | Severity Range | Current Status | Intervention Window |
|---|---|---|---|---|
| Current Active | 2020-2024 | Medium-High | Multiple risks active | Closing rapidly |
| Near-term Critical | 2025-2027 | High-Extreme | Approaching thresholds | Open but narrowing |
| Long-term Existential | 2030-2050+ | Extreme-Catastrophic | Early warning signs | Wide but requires early action |
| Cascade Effects | Ongoing | Amplifies all categories | Accelerating | Immediate intervention needed |
Risk Activation Framework
Activation Criteria
| Criterion | Description | Example Threshold |
|---|---|---|
| Capability Crossing | AI can perform necessary tasks | GPT-4 level code generation for cyberweapons |
| Deployment Context | Systems deployed in relevant settings | Autonomous agents with internet access |
| Barrier Erosion | Technical/social barriers removed | Open-source parity reducing control |
| Incentive Alignment | Actors motivated to exploit | Economic pressure + accessible tools |
Progress Tracking Methodology
We assess progress toward activation using:
- Technical benchmarks from evaluation organizationsOrganizationMETRMETR conducts pre-deployment dangerous capability evaluations for frontier AI labs (OpenAI, Anthropic, Google DeepMind), testing autonomous replication, cybersecurity, CBRN, and manipulation capabi...Quality: 66/100
- Deployment indicators from major AI labs
- Adversarial use cases documented in security research
- Expert opinionAi Transition Model MetricExpert OpinionComprehensive analysis of expert beliefs on AI risk shows median 5-10% P(doom) but extreme disagreement (0.01-99% range), with AGI forecasts compressing from 50+ years (2020) to ~5 years (2024). De...Quality: 61/100 surveys on capability timelines
Current Risks (Already Active)
Category: Misuse Risks
| Risk | Status | Current Evidence | Impact Scale | Source |
|---|---|---|---|---|
| DisinformationRiskAI DisinformationPost-2024 analysis shows AI disinformation had limited immediate electoral impact (cheap fakes used 7x more than AI content), but creates concerning long-term epistemic erosion with 82% higher beli...Quality: 54/100 at scale | Active | 2024 election manipulation campaigns | $1-10B annual | Reuters↗🔗 web★★★★☆ReutersReuterstimelinecapabilityrisk-assessmentSource ↗ |
| Spear phishing enhancement | Active | 82% higher believability vs human-written | $10B+ annual losses | IBM Security↗🔗 webIBM Securitycybersecuritytimelinecapabilityrisk-assessmentSource ↗ |
| Code vulnerability exploitation | Partially active | GPT-4 identifies 0-days, limited autonomy | Medium severity | AnthropicOrganizationAnthropicComprehensive profile of Anthropic, founded in 2021 by seven former OpenAI researchers (Dario and Daniela Amodei, Chris Olah, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish) with early funding... evals↗🔗 web★★★★☆AnthropicAnthropic evalsevaluationtimelinecapabilityrisk-assessmentSource ↗ |
| Academic fraud | Active | 30-60% of student submissions flagged | Education integrity crisis | Stanford study↗🔗 web★★★★☆Stanford HAIStanford studytimelinecapabilityrisk-assessmentSource ↗ |
| Romance/financial scams | Active | AI voice cloning in elder fraud | $1B+ annual | FTC reports↗🏛️ government★★★★☆Federal Trade CommissionFTC reportstimelinecapabilityrisk-assessmentSource ↗ |
Category: Structural Risks
| Risk | Status | Current Evidence | Impact Scale | Trend |
|---|---|---|---|---|
| Epistemic erosionRiskEpistemic CollapseEpistemic collapse describes the complete erosion of society's ability to establish factual consensus when AI-generated synthetic content overwhelms verification capacity. Current AI detectors achi...Quality: 49/100 | Active | 40% decline in information trust | Society-wide | Accelerating |
| Economic displacement | Beginning | 15% of customer service roles automated | 200M+ jobs at risk | Expanding |
| Attention manipulation | Active | Algorithm-driven engagement optimization | Mental health crisis | Intensifying |
| Dependency formation | Active | 60% productivity loss when tools unavailable | Skill atrophy beginning | Growing |
Category: Technical Risks
| Risk | Status | Current Evidence | Mitigation Level | Progress |
|---|---|---|---|---|
| Reward hackingRiskReward HackingComprehensive analysis showing reward hacking occurs in 1-2% of OpenAI o3 task attempts, with 43x higher rates when scoring functions are visible. Mathematical proof establishes it's inevitable for...Quality: 91/100 | Active | Documented in all RLHFCapabilityRLHFRLHF/Constitutional AI achieves 82-85% preference improvements and 40.8% adversarial attack reduction for current systems, but faces fundamental scalability limits: weak-to-strong supervision shows...Quality: 63/100 systems | Partial guardrails | No clear progress |
| SycophancyRiskSycophancySycophancy—AI systems agreeing with users over providing accurate information—affects 34-78% of interactions and represents an observable precursor to deceptive alignment. The page frames this as a...Quality: 65/100 | Active | Models agree with user regardless of truth | Research stage | Limited progress |
| Prompt injection | Active | Jailbreaks succeed >50% of time | Defense research ongoing | Cat-mouse game |
| Hallucination/confabulation | Active | 15-30% false information in outputs | Detection tools emerging | Gradual improvement |
Near-Term Risks (2025-2027 Activation Window)
Critical Misuse Risks
| Risk | Activation Window | Key Threshold | Current Progress | Intervention Status |
|---|---|---|---|---|
| BioweaponsRiskBioweapons RiskComprehensive synthesis of AI-bioweapons evidence through early 2026, including the FRI expert survey finding 5x risk increase from AI capabilities (0.3% → 1.5% annual epidemic probability), Anthro...Quality: 91/100 uplift | 2025-2028 | Synthesis guidance beyond textbooks | 60-80% to threshold | Active screening efforts↗🔗 webActive screening effortstimelinecapabilityrisk-assessmentSource ↗ |
| CyberweaponRiskCyberweapons RiskComprehensive analysis showing AI-enabled cyberweapons represent a present, high-severity threat with GPT-4 exploiting 87% of one-day vulnerabilities at $8.80/exploit and the first documented AI-or...Quality: 91/100 development | 2025-2027 | Autonomous 0-day discovery | 70-85% to threshold | Limited defensive preparation |
| PersuasionCapabilityPersuasion and Social ManipulationGPT-4 achieves superhuman persuasion in controlled settings (64% win rate, 81% higher odds with personalization), with AI chatbots demonstrating 4x the impact of political ads (3.9 vs ~1 point vote...Quality: 63/100 weapons | 2025-2026 | Personalized, adaptive manipulation | 80-90% to threshold | No systematic defenses |
| Mass deepfake attacks | Active-2026 | Real-time, undetectable generation | 85-95% to threshold | Detection research lagging↗🔗 webDetection research laggingtimelinecapabilityrisk-assessmentSource ↗ |
Control and Alignment Risks
| Risk | Activation Window | Key Threshold | Current Progress | Research Investment |
|---|---|---|---|---|
| Agentic systemCapabilityAgentic AIAnalysis of agentic AI capabilities and deployment challenges, documenting industry forecasts (40% of enterprise apps by 2026, $199B market by 2034) alongside implementation difficulties (40%+ proj... failures | 2025-2026 | Multi-step autonomous task execution | 70-80% to threshold | $500M+ annually |
| Situational awarenessCapabilitySituational AwarenessComprehensive analysis of situational awareness in AI systems, documenting that Claude 3 Opus fakes alignment 12% baseline (78% post-RL), 5 of 6 frontier models demonstrate scheming capabilities, a...Quality: 67/100 | 2025-2027 | Strategic self-modeling capability | 50-70% to threshold | Research accelerating |
| SandbaggingRiskAI Capability SandbaggingSystematically documents sandbagging (strategic underperformance during evaluations) across frontier models, finding 70-85% detection accuracy with white-box probes, 18-24% accuracy drops on autono...Quality: 67/100 on evals | 2026-2028 | Concealing capabilities from evaluators | 40-60% to threshold | Limited detection work |
| Human oversight evasion | 2026-2029 | Identifying and exploiting oversight gaps | 30-50% to threshold | Control research beginning |
Structural Transformation Risks
| Risk | Activation Window | Key Threshold | Economic Impact | Policy Preparation |
|---|---|---|---|---|
| Mass unemployment crisis | 2026-2030 | >10% of jobs automatable within 2 years | $5-15T GDP impact | Minimal policy frameworks |
| Authentication collapseRiskAuthentication CollapseComprehensive synthesis showing human deepfake detection has fallen to 24.5% for video and 55% overall (barely above chance), with AI detectors dropping from 90%+ to 60% on novel fakes. Economic im...Quality: 57/100 | 2025-2027 | Can't distinguish human vs AI content | Democratic processes at risk | Technical solutions emerging↗🔗 webC2PA Explainer VideosThe Coalition for Content Provenance and Authenticity (C2PA) offers a technical standard that acts like a 'nutrition label' for digital content, tracking its origin and edit his...epistemictimelineauthenticationcapability+1Source ↗ |
| AI-powered surveillance state | 2025-2028 | Real-time behavior prediction | Human rights implications | Regulatory gaps |
| Expertise atrophyRiskAI-Induced Expertise AtrophyExpertise atrophy—humans losing skills to AI dependence—poses medium-term risks across critical domains (aviation, medicine, programming), creating oversight failures when AI errs or fails. Evidenc...Quality: 65/100 | 2026-2032 | Human skills erode from AI dependence | Innovation capacity loss | No systematic response |
Long-Term Risks (ASI-Level Requirements)
Existential Risk Category
| Risk | Estimated Window | Key Capability Threshold | Confidence Level | Research Investment |
|---|---|---|---|---|
| Misaligned superintelligenceAi Transition Model ScenarioMisaligned Catastrophe - The Bad EndingComprehensive scenario analysis of AI misalignment catastrophe, synthesizing expert probability estimates (5-14.4% median/mean extinction risk by 2100) with 2024-2025 empirical evidence of alignmen...Quality: 64/100 | 2030-2050+ | Systems exceed human-level at alignment-relevant tasks | Very Low | $1B+ annually |
| Recursive self-improvementCapabilitySelf-Improvement and Recursive EnhancementComprehensive analysis of AI self-improvement from current AutoML systems (23% training speedups via AlphaEvolve) to theoretical intelligence explosion scenarios, with expert consensus at ~50% prob...Quality: 69/100 | 2030-2045+ | AI meaningfully improves AI architecture | Low | Limited research |
| Decisive strategic advantage | 2030-2040+ | Single actor gains insurmountable technological lead | Low | Policy research only |
| Irreversible value lock-inParameterValue Lock-inThis page contains only placeholder React components with no actual content about value lock-in scenarios or their implications for AI risk prioritization. | 2028-2040+ | Permanent commitment to suboptimal human values | Low-Medium | Philosophy/governance research |
Advanced Deception and Control
| Risk | Estimated Window | Capability Requirement | Detection Difficulty | Mitigation Research |
|---|---|---|---|---|
| Strategic deceptionRiskSchemingScheming—strategic AI deception during training—has transitioned from theoretical concern to observed behavior across all major frontier models (o1: 37% alignment faking, Claude: 14% harmful compli...Quality: 74/100 | 2027-2035 | Model training dynamics and hide intentions | Very High | Interpretability researchSafety AgendaInterpretabilityMechanistic interpretability has extracted 34M+ interpretable features from Claude 3 Sonnet with 90% automated labeling accuracy and demonstrated 75-85% success in causal validation, though less th...Quality: 66/100 |
| Coordinated AI systems | 2028-2040 | Multiple AI systems coordinate against humans | High | Multi-agent safetyApproachMulti-Agent SafetyMulti-agent safety addresses coordination failures, conflict, and collusion risks when AI systems interact. A 2025 report from 50+ researchers identifies seven key risk factors; empirical studies s...Quality: 68/100 research |
| Large-scale human manipulation | 2028-2035 | Accurate predictive models of human behavior | Medium | Social science integration |
| Critical infrastructure control | 2030-2050+ | Simultaneous control of multiple key systems | Very High | Air-gapped research |
Risk Interaction and Cascade Effects
Cascade Amplification Matrix
| Triggering Risk | Amplifies | Mechanism | Timeline Impact |
|---|---|---|---|
| Disinformation proliferationRiskAI ProliferationAI proliferation accelerated dramatically as the capability gap narrowed from 18 to 6 months (2022-2024), with open-source models like DeepSeek R1 now matching frontier performance. US export contr...Quality: 60/100 | Epistemic collapseRiskEpistemic CollapseEpistemic collapse describes the complete erosion of society's ability to establish factual consensus when AI-generated synthetic content overwhelms verification capacity. Current AI detectors achi...Quality: 49/100 | Trust erosion accelerates | -1 to -2 years |
| Cyberweapon autonomy | Authentication collapseRiskAuthentication CollapseComprehensive synthesis showing human deepfake detection has fallen to 24.5% for video and 55% overall (barely above chance), with AI detectors dropping from 90%+ to 60% on novel fakes. Economic im...Quality: 57/100 | Digital infrastructure vulnerability | -1 to -3 years |
| Bioweapons accessibility | Authoritarian control | Crisis enables power concentration | Variable |
| Economic displacement | Social instability | Reduces governance capacity | -0.5 to -1.5 years |
| Any major AI incident | Regulatory capture | Crisis mode enables bad policy | -2 to -5 years |
Acceleration Factors
| Factor | Timeline Impact | Probability by 2027 | Evidence |
|---|---|---|---|
| Algorithmic breakthrough | -1 to -3 years across categories | 15-30% | Historical ML progress |
| 10x compute scaling | -0.5 to -1.5 years | 40-60% | Current compute trends↗🔗 web★★★★☆Epoch AICurrent compute trendscomputetimelinecapabilityrisk-assessmentSource ↗ |
| Open-source capability parity | -1 to -2 years on misuse risks | 50-70% | Open model progress↗🔗 webModel weight leaderboardsrisk-factordiffusioncontroltimeline+1Source ↗ |
| Geopolitical AI arms race | -0.5 to -2 years overall | 30-50% | US-China competition intensifying |
| Major safety failure/incident | Variable, enables governance | 20-40% | Base rate of tech failures |
Deceleration Factors
| Factor | Timeline Impact | Probability by 2030 | Feasibility |
|---|---|---|---|
| Scaling laws plateau | +2 to +5 years | 15-30% | Some evidence emerging |
| Strong international AI governanceParameterAI GovernanceThis page contains only component imports with no actual content - it displays dynamically loaded data from an external source that cannot be evaluated. | +1 to +3 years on misuse | 10-20% | Limited progress so far |
| Major alignment breakthrough | Variable positive impact | 10-25% | Research uncertainty high |
| Physical compute constraints | +0.5 to +2 years | 20-35% | Semiconductor bottlenecks |
| Economic/energy limitations | +1 to +3 years | 15-25% | Training cost growth |
Critical Intervention Windows
Time-Sensitive Priority Matrix
| Risk Category | Window Opens | Window Closes | Intervention Cost | Effectiveness if Delayed |
|---|---|---|---|---|
| Bioweapons screening | 2020 (missed) | 2027 | $500M-1B | 50% reduction |
| Cyber defensive AI | 2023 | 2026 | $1-3B | 70% reduction |
| Authentication infrastructure | 2024 | 2026 | $300-600M | 30% reduction |
| AI control research | 2022 | 2028 | $1-2B annually | 20% reduction |
| International governance | 2023 | 2027 | $200-500M | 80% reduction |
| Alignment foundations | 2015 | 2035+ | $2-5B annually | Variable |
Leverage Analysis by Intervention Type
| Intervention Category | Current Leverage | Peak Leverage Window | Investment Required | Expected Impact |
|---|---|---|---|---|
| DNA synthesis screening | High | 2024-2027 | $100-300M globally | Delays bio threshold 2-3 years |
| Model evaluation standards | Medium | 2024-2026 | $50-150M annually | Enables risk detection |
| Interpretability breakthroughs | Very High | 2024-2030 | $500M-1B annually | Addresses multiple long-term risks |
| Defensive cyber-AI | Medium | 2024-2026 | $1-2B | Extends defensive advantage |
| Public authentication systems | High | 2024-2026 | $200-500M | Preserves epistemic infrastructureApproachAI-Era Epistemic InfrastructureComprehensive analysis of epistemic infrastructure showing AI fact-checking achieves 85-87% accuracy at $0.10-$1.00 per claim versus $50-200 for human verification, while Community Notes reduces mi...Quality: 59/100 |
| International AI treaties | Very High | 2024-2027 | $100-200M | Sets precedent for future governance |
Probability Calibration Over Time
Risk Activation Probabilities by Year
| Risk Category | 2025 | 2027 | 2030 | 2035 | 2040 |
|---|---|---|---|---|---|
| Mass disinformation | 95% (active) | 99% | 99% | 99% | 99% |
| Bioweapons uplift (meaningful) | 25% | 50% | 70% | 85% | 95% |
| Autonomous cyber operations | 40% | 75% | 90% | 99% | 99% |
| Large-scale job displacement | 15% | 40% | 65% | 85% | 95% |
| Authentication crisis | 30% | 60% | 80% | 95% | 99% |
| Agentic AI control failures | 35% | 70% | 90% | 99% | 99% |
| Meaningful situational awareness | 20% | 50% | 75% | 90% | 95% |
| Strategic AI deception | 5% | 20% | 45% | 70% | 85% |
| ASI-level misalignment | <1% | 3% | 15% | 35% | 55% |
Uncertainty Ranges and Expert Disagreement
| Risk | Optimistic Timeline | Median | Pessimistic Timeline | Expert Confidence |
|---|---|---|---|---|
| Cyberweapon autonomy | 2028-2030 | 2025-2027 | 2024-2025 | Medium (70% within range) |
| Bioweapons threshold | 2030-2035 | 2026-2029 | 2024-2026 | Low (50% within range) |
| Mass unemployment | 2035-2040 | 2028-2032 | 2025-2027 | Very Low (30% within range) |
| Superintelligence | 2045-Never | 2030-2040 | 2027-2032 | Very Low (20% within range) |
Strategic Resource Allocation
Investment Priority Framework
| Priority Tier | Timeline | Investment Level | Rationale |
|---|---|---|---|
| Tier 1: Critical | Immediate-2027 | $3-5B annually | Window closing rapidly |
| Tier 2: Important | 2025-2030 | $1-2B annually | Foundation for later risks |
| Tier 3: Foundational | 2024-2035+ | $500M-1B annually | Long-term preparation |
Recommended Investment Allocation
| Research Area | Annual Investment | Justification | Expected ROI |
|---|---|---|---|
| Bioweapons screening infrastructure | $200-400M (2024-2027) | Critical window closing | Very High - prevents catastrophic risk |
| AI interpretability research | $300-600M ongoing | Multi-risk mitigation | High - enables control across scenarios |
| Cyber-defense AI systems | $500M-1B (2024-2026) | Maintaining defensive advantage | Medium-High |
| Authentication/verification tech | $100-200M (2024-2026) | Preserving epistemic infrastructure | High |
| International governance capacity | $100-200M (2024-2027) | Coordination before crisis | Very High - prevents race dynamics |
| AI control methodology | $400-800M ongoing | Bridge to long-term safety | High |
| Economic transition planning | $200-400M (2024-2030) | Social stability preservation | Medium |
Key Cruxes and Uncertainties
Timeline Uncertainty Analysis
| Core Uncertainty | If Optimistic | If Pessimistic | Current Best Estimate | Implications |
|---|---|---|---|---|
| Scaling law continuation | Plateau by 2027-2030 | Continue through 2035+ | 60% likely to continue | ±3 years on all timelines |
| Open-source capability gap | Maintains 2+ year lag | Achieves parity by 2026 | 55% chance of rapid catch-up | ±2 years on misuse risks |
| Alignment research progress | Major breakthrough by 2030 | Limited progress through 2035 | 20% chance of breakthrough | ±5-10 years on existential risk |
| Geopolitical cooperation | Successful AI treaties | Intensified arms race | 25% chance of cooperation | ±2-5 years on multiple risks |
| Economic adaptation speed | Smooth transition over 10+ years | Rapid displacement over 3-5 years | 40% chance of rapid displacement | Social stability implications |
Research and Policy Dependencies
| Dependency | Success Probability | Impact if Failed | Mitigation Options |
|---|---|---|---|
| International bioweapons screening | 60% | Bioweapons threshold advances 2-3 years | National screening systems, detection research |
| AI evaluationApproachAI EvaluationComprehensive overview of AI evaluation methods spanning dangerous capability assessment, safety properties, and deception detection, with categorized frameworks from industry (Anthropic Constituti...Quality: 72/100 standardization | 40% | Reduced early warning capability | Industry self-regulation, government mandates |
| Interpretability breakthroughs | 30% | Limited control over advanced systems | Multiple research approaches, AI-assisted research |
| Democratic governance adaptation | 35% | Poor quality regulation during crisis | Early capacity building, expert networks |
Implications for Different Stakeholders
For AI Development Organizations
Immediate priorities (2024-2025):
- Implement robust evaluations for near-term risksCruxAI Accident Risk CruxesComprehensive survey of AI safety researcher disagreements on accident risks, quantifying probability ranges for mesa-optimization (15-55%), deceptive alignment (15-50%), and P(doom) (5-35% median ...Quality: 67/100
- Establish safety teams scaling with capability teams
- Contribute to industry evaluation standards
Near-term preparations (2025-2027):
- Deploy monitoring systems for newly activated risks
- Engage constructively in governance frameworks
- Research control methods before needed
For Policymakers
Critical window actions:
- Establish regulatory frameworks before crisis mode
- Focus on near-term risks to build governance credibility
- Invest in international coordination mechanismsPolicyInternational Coordination MechanismsComprehensive analysis of international AI coordination mechanisms shows growing but limited progress: 11-country AI Safety Institute network with ~$200M budget expanding to include India; Council ...Quality: 91/100
Priority areas:
- Bioweapons screening infrastructure
- AI evaluation and monitoring standards
- Economic transition support systems
- Authentication and verification requirements
For Safety Researchers
Optimal portfolio allocation:
- 40% near-term (1-2 generation) risk mitigation
- 40% foundational research for long-term risks
- 20% current risk mitigation and response
High-leverage research areas:
- InterpretabilitySafety AgendaInterpretabilityMechanistic interpretability has extracted 34M+ interpretable features from Claude 3 Sonnet with 90% automated labeling accuracy and demonstrated 75-85% success in causal validation, though less th...Quality: 66/100 for multiple risk categories
- AI controlSafety AgendaAI ControlAI Control is a defensive safety approach that maintains control over potentially misaligned AI through monitoring, containment, and redundancy, offering 40-60% catastrophic risk reduction if align...Quality: 75/100 methodology development
- Evaluation methodology for emerging capabilities
- Social science integration for structural risks
For Civil Society Organizations
Advocacy priorities:
- Demand transparency in capability evaluations
- Push for public interest representation in governance
- Support authentication infrastructure development
- Advocate for economic transition policies
Limitations and Model Uncertainty
Methodological Limitations
| Limitation | Impact on Accuracy | Mitigation Strategies |
|---|---|---|
| Expert overconfidence | Timelines may be systematically early/late | Multiple forecasting methods, base rate reference |
| Capability discontinuities | Sudden activation possible | Broader uncertainty ranges, multiple scenarios |
| Interaction complexity | Cascade effects poorly understood | Systems modeling, historical analogies |
| Adversarial adaptation | Defenses may fail faster than expected | Red team exercises, worst-case planning |
Areas for Model Enhancement
- Better cascade modeling - More sophisticated interaction effects
- Adversarial dynamics - How attackers adapt to defenses
- Institutional response capacity - How organizations adapt to new risks
- Cross-cultural variation - Risk manifestation in different contexts
- Economic feedback loops - How risk realization affects development
Sources & Resources
Primary Research Sources
| Organization | Type | Key Contributions |
|---|---|---|
| Anthropic↗🔗 web★★★★☆AnthropicAnthropic evalsevaluationtimelinecapabilityrisk-assessmentSource ↗ | AI Lab | Risk evaluation methodologies, scaling policies |
| OpenAIOrganizationOpenAIComprehensive organizational profile of OpenAI documenting evolution from 2015 non-profit to commercial AGI developer, with detailed analysis of governance crisis, safety researcher exodus (75% of ...↗🔗 web★★★★☆OpenAIOpenAItimelinecapabilityrisk-assessmenttraining+1Source ↗ | AI Lab | Preparedness framework, capability assessment |
| METR↗🔗 web★★★★☆METRmetr.orgsoftware-engineeringcode-generationprogramming-aisocial-engineering+1Source ↗ | Evaluation Org | Technical capability evaluations |
| RAND Corporation↗🔗 web★★★★☆RAND CorporationRAND: AI and National Securitycybersecurityagenticplanninggoal-stability+1Source ↗ | Think Tank | Policy analysis, national security implications |
| Center for AI SafetyOrganizationCenter for AI SafetyCAIS is a research organization that has distributed $2M+ in compute grants to 200+ researchers, published 50+ safety papers including benchmarks adopted by Anthropic/OpenAI, and organized the May ...Quality: 42/100↗🔗 web★★★★☆Center for AI SafetyCAIS SurveysThe Center for AI Safety conducts technical and conceptual research to mitigate potential catastrophic risks from advanced AI systems. They take a comprehensive approach spannin...safetyx-risktalentfield-building+1Source ↗ | Safety Org | Risk taxonomy, expert opinion surveys |
Academic Literature
| Paper | Authors | Key Finding |
|---|---|---|
| Model evaluation for extreme risks↗📄 paper★★★☆☆arXivModel Evaluation for Extreme RisksToby Shevlane, Sebastian Farquhar, Ben Garfinkel et al. (2023)alignmentgovernancecapabilitiessafety+1Source ↗ | Anthropic Constitutional AIApproachConstitutional AIConstitutional AI is Anthropic's methodology using explicit principles and AI-generated feedback (RLAIF) to train safer models, achieving 3-10x improvements in harmlessness while maintaining helpfu...Quality: 70/100 Team | Evaluation frameworks for dangerous capabilities |
| AI timelines and capabilities↗📄 paper★★★☆☆arXivAI timelines and capabilitiesDeepSeek-AI, :, Xiao Bi et al. (2024)capabilitiestrainingevaluationopen-source+1Source ↗ | Various forecasting research | Capability development trajectories |
| Cybersecurity implications of AI↗🔗 web★★★★☆CSET GeorgetownCybersecurity implications of AIcybersecuritytimelinecapabilityrisk-assessmentSource ↗ | CSETOrganizationCSET (Center for Security and Emerging Technology)CSET is a $100M+ Georgetown center with 50+ staff conducting data-driven AI policy research, particularly on U.S.-China competition and export controls. The center conducts hundreds of annual gover...Quality: 43/100 | Near-term cyber risk assessment |
Policy and Governance Sources
Expert Opinion and Forecasting
Related Models and Cross-References
Complementary Risk Models
- AI Capability Threshold ModelModelAI Capability Threshold ModelComprehensive framework mapping AI capabilities across 5 dimensions to specific risk thresholds, finding authentication collapse/mass persuasion risks at 70-85% likelihood by 2027, bioweapons devel...Quality: 72/100 - Specific capability requirements for risk activation
- Bioweapons AI Uplift ModelModelAI Uplift Assessment ModelQuantitative assessment estimating AI provides modest knowledge uplift for bioweapons (1.0-1.2x per RAND 2024) but concerning evasion capabilities (2-3x, potentially 7-10x by 2028), with projected ...Quality: 70/100 - Detailed biological weapons timeline
- Cyberweapons Attack AutomationModelAutonomous Cyber Attack TimelineThis model projects AI achieving fully autonomous cyber attack capability (Level 4) by 2029-2033, with current systems at ~50% progress and Level 3 attacks already documented in September 2025. Pro...Quality: 63/100 - Cyber capability development
- Authentication Collapse TimelineModelAuthentication Collapse Timeline ModelProjects when AI-generated content becomes undetectable across modalities: text detection already at ~50% (random chance), images declining 5-10% annually toward 2026-2028 failure, audio/video foll...Quality: 59/100 - Digital verification crisis
- Economic Disruption Impact - Labor market transformation
Risk Category Cross-References
- Accident Risks - Technical AI safety failures
- Misuse Risks - Intentional harmful applications
- Structural Risks - Systemic societal impacts
- Epistemic Risks - Information environment degradation
Response Strategy Integration
- Governance Responses - Policy intervention strategies
- Technical Safety Research - Engineering solutions
- International Coordination - Global cooperation frameworks