Capability Threshold Model

Overview

Different AI risks require different capability levels to become dangerous. A system that can write convincing phishing emails poses different risks than one that can autonomously discover zero-day vulnerabilities. This model maps specific capability requirements to specific risks, helping predict when risks activate as capabilities improve.

The capability threshold model provides a structured framework for understanding how AI systems transition from relatively benign to potentially dangerous across multiple risk domains. Rather than treating AI capability as a single dimension or risks as uniformly dependent on general intelligence, this model recognizes that specific risks emerge when systems cross particular capability thresholds in relevant dimensions. According to the International AI Safety Report (October 2025)↗🔗 webInternational AI Safety Report (October 2025)This is an official interim update to the landmark International AI Safety Report, a major intergovernmental reference document on AI safety risks; highly relevant for those tracking global AI governance and the evolving capabilities landscape as of late 2025.A focused interim update to the International AI Safety Report, chaired by Yoshua Bengio, covering significant developments in AI capabilities and their risk implications betwee...ai-safetygovernancecapabilitiespolicy+4Source ↗, governance choices in 2025-2026 must internalize that capability scaling has decoupled from parameter count, meaning risk thresholds can be crossed between annual cycles.

Key findings include 15-25% benchmark performance indicating early risk emergence, 50% marking qualitative shifts to complex autonomous execution, and most critical thresholds estimated to cross between 2025-2029 across misuse, control, and structural risk categories. The Future of Life InstituteOrganizationFuture of Life InstituteComprehensive profile of FLI documenting $25M+ in grants distributed (2015: $7M to 37 projects, 2021: $25M program), major public campaigns (Asilomar Principles with 5,700+ signatories, 2023 Pause ...Quality: 46/100's 2025 AI Safety Index↗🔗 web★★★☆☆Future of Life InstituteFLI AI Safety Index Summer 2025Published by the Future of Life Institute, this index provides a structured external audit of major AI labs' safety practices, useful for tracking industry accountability trends and identifying gaps between stated safety commitments and measurable actions.The Future of Life Institute's AI Safety Index Summer 2025 systematically evaluates leading AI companies on safety practices, finding widespread deficiencies across risk managem...ai-safetygovernanceevaluationexistential-risk+4Source ↗ reveals an industry struggling to keep pace with its own rapid capability advances, with companies claiming AGI achievement within the decade yet none scoring above D in existential safety planning.

Risk Impact Assessment

Capability Dimensions Framework

AI capabilities decompose into five distinct dimensions that progress at different rates. Understanding these separately is crucial because different risks require different combinations. According to Epoch AIOrganizationEpoch AIEpoch AI maintains comprehensive databases tracking 3,200+ ML models showing 4.4x annual compute growth and projects data exhaustion 2026-2032. Their empirical work directly informed EU AI Act's 10...Quality: 51/100's tracking↗🔗 web★★★★☆Epoch AIEpoch AI: AI Trends & Metrics DashboardEpoch AI's trends dashboard is a key empirical reference for AI safety researchers assessing capability trajectories and compute scaling; useful for grounding risk assessments and policy arguments in observed data.Epoch AI's trends page provides data-driven tracking of key metrics in AI development, including compute scaling, model capabilities, and training trends. It serves as a quantit...capabilitiescomputeevaluationai-safety+2Source ↗, the training compute of frontier AI models has grown by 5x per year since 2020, and the Epoch Capabilities Index shows frontier model improvement nearly doubled in 2024, from ~8 points/year to ~15 points/year.

flowchart TD
  subgraph DIMS["Capability Dimensions"]
      DK[Domain Knowledge] --> RISK
      RD[Reasoning Depth] --> RISK
      PH[Planning Horizon] --> RISK
      SM[Strategic Modeling] --> RISK
      AE[Autonomous Execution] --> RISK
  end

  subgraph RISK["Risk Activation Thresholds"]
      AUTH[Authentication Collapse<br/>Threshold: 2025-2027]
      BIO[Bioweapons Uplift<br/>Threshold: 2026-2029]
      CYBER[Cyberweapons<br/>Threshold: 2025-2027]
      SCHEME[Strategic Deception<br/>Threshold: 2027-2035+]
  end

  RISK --> AUTH
  RISK --> BIO
  RISK --> CYBER
  RISK --> SCHEME

  style AUTH fill:#ffcccc
  style BIO fill:#ffcccc
  style CYBER fill:#ffddcc
  style SCHEME fill:#ffe6cc

Domain Knowledge Benchmarks

Current measurement approaches show significant gaps in assessing practical domain expertise:

Assessment quality reflects how well benchmarks capture practical expertise versus academic knowledge.

Reasoning Depth Progression

The ARC Prize 2024-2025 results↗🔗 webARC Prize 2024-2025 resultsARC Prize is a major AI benchmark competition testing abstract reasoning; its results are closely watched by the AI safety community as a potential indicator of progress toward AGI-level capabilities and associated risk thresholds.Comprehensive analysis of the ARC Prize competition results for 2024-2025, evaluating AI systems' performance on the Abstraction and Reasoning Corpus (ARC) benchmark designed to...capabilitiesevaluationai-safetytechnical-safety+1Source ↗ demonstrate the critical threshold zone for complex reasoning. On ARC-AGI-1, OpenAI's o3-preview achieved 75.7% accuracy (near human level of 98%), while on the harder ARC-AGI-2 benchmark, even advanced models score only single-digit percentages, yet humans can solve every task.

Recent breakthrough (December 2025): Poetiq with GPT-5.2 X-High↗🔗 webARC Prize 2024-2025 resultsARC Prize is a major AI benchmark competition testing abstract reasoning; its results are closely watched by the AI safety community as a potential indicator of progress toward AGI-level capabilities and associated risk thresholds.Comprehensive analysis of the ARC Prize competition results for 2024-2025, evaluating AI systems' performance on the Abstraction and Reasoning Corpus (ARC) benchmark designed to...capabilitiesevaluationai-safetytechnical-safety+1Source ↗ achieved 75% on ARC-AGI-2, surpassing the average human test-taker score of 60% for the first time, demonstrating rapid progress on complex reasoning tasks.

Risk-Capability Mapping

Near-Term Risks (2025-2027)

Authentication Collapse

The volume of deepfakes has grown explosively: Deloitte's 2024 analysis↗🔗 webDeloitte's 2024 analysisIndustry analyst report from Deloitte relevant to AI governance and deployment policy discussions; useful for understanding economic and societal implications of generative AI misuse, particularly around synthetic media and content authentication standards.Deloitte's 2024 analysis frames deepfakes as a cybersecurity-scale threat to online trust, projecting the deepfake detection market will grow 42% annually from $5.5B in 2023 to ...governancedeploymentpolicycapabilities+3Source ↗ estimates growth from roughly 500,000 online deepfakes in 2023 to about 8 million in 2025, with annual growth nearing 900%. Voice cloning has crossed what experts call the "indistinguishable threshold"--a few seconds of audio now suffice to generate a convincing clone.

Key Threshold Capabilities:

• Generate synthetic content indistinguishable from authentic across all modalities
• Real-time interactive video generation (NVIDIA Omniverse↗🔗 webNVIDIA Omniverse PlatformOmniverse is primarily an industrial/commercial platform but is relevant to AI safety discussions around simulation-based training, digital twins, and the compute infrastructure underpinning advanced AI capabilities in robotics and autonomous systems.NVIDIA Omniverse is a platform for building and operating metaverse applications, enabling real-time 3D simulation, collaboration, and digital twin creation. It provides tools f...capabilitiesdeploymentcomputeevaluation+1Source ↗)
• Defeat detection systems designed to identify AI content
• Mimic individual styles from minimal samples

Detection Challenges: OpenAI's deepfake detection tool↗🔗 webOpenAI's deepfake detection toolUseful for practitioners and policy researchers tracking the state of deepfake countermeasures; relevant to AI misuse risk assessment and content authenticity governance discussions.This resource surveys leading AI-powered deepfake detection tools available in 2025, including OpenAI's detection tool, evaluating their capabilities for identifying synthetical...capabilitiesdeploymentevaluationred-teaming+3Source ↗ identifies DALL-E 3 images with 98.8% accuracy but only flags 5-10% of images from other AI tools. Multi-modal attacks combining deepfaked video, synthetic voices, and fabricated documents are increasing.

Current Status: OpenAI's Sora↗🔗 web★★★★☆OpenAISora – OpenAI Text-to-Video Generation AppSora is relevant to AI safety discussions around capabilities proliferation, deepfake risks, misuse potential of generative video, and the pace of frontier model deployment by major labs.Sora is OpenAI's text-to-video generation model and app that converts text prompts or images into videos with high realism, including automatic sound. It supports features like ...capabilitiesdeploymentmultimodalgenerative-ai+2Source ↗ and Meta's Make-A-Video↗🔗 webMeta Make-A-Video: Text-to-Video AI SystemA demonstration of Meta's text-to-video AI capability, relevant to AI safety discussions around synthetic media risks, capability thresholds, and the governance challenges posed by increasingly realistic generative AI systems.Meta's Make-A-Video is an AI system that generates short video clips from text descriptions, images, or existing videos. It extends text-to-image generation techniques into the ...capabilitiesdeploymentgovernancerisk-assessment+2Source ↗ demonstrate near-threshold video generation. ElevenLabs↗🔗 webElevenLabs - AI Voice Generation PlatformElevenLabs is a major commercial AI voice platform relevant to AI safety discussions around synthetic media, voice cloning misuse, deepfake audio, and the governance challenges posed by increasingly accessible identity-spoofing technologies.ElevenLabs is a leading AI voice technology platform offering text-to-speech, voice cloning, speech-to-text, and AI agent capabilities across 70+ languages. It serves enterprise...capabilitiessynthetic-mediadeploymentgovernance+2Source ↗ achieves voice cloning from <30 seconds of audio.

Mass Persuasion Capabilities

Research Evidence:

• Anthropic (2024)↗🔗 web★★★★☆AnthropicClaude 3 Model Card (Anthropic, 2024)Official Anthropic model card for Claude 3; key reference for understanding how a leading AI lab evaluates and communicates safety properties and capability thresholds for a frontier model under a formal responsible scaling policy.Anthropic's official model card for the Claude 3 family (Haiku, Sonnet, Opus), documenting capability evaluations, safety assessments, and alignment properties. It covers fronti...capabilitiesevaluationred-teamingdeployment+5Source ↗ shows Claude 3 achieves 84% on psychology benchmarks
• Stanford HAI study↗🔗 web★★★★☆Stanford HAIHumans Are More Likely to Believe Messages from AI (Stanford HAI)Relevant to AI safety discussions around misuse, disinformation, and the social risks of deploying persuasive AI systems; underscores why transparency and content labeling policies matter.A Stanford HAI study examines how people respond to messages they believe are generated by AI versus humans, finding that individuals tend to place higher credibility or trust i...ai-safetydeploymentgovernancepolicy+3Source ↗ finds AI-generated content 82% higher believability
• MIT persuasion study↗📄 paper★★★★★Science (peer-reviewed)MIT persuasion studyEmpirical study examining humans' ability to distinguish AI-generated content (GPT-3) from authentic information and detect disinformation, directly relevant to understanding AI capabilities in content generation and information integrity risks.G. Spitale, N. Biller-Andorno, Federico Germani (2023)220 citationsThis MIT study examined whether humans can distinguish between accurate and false information in tweets, and whether they can identify AI-generated content from GPT-3 versus hum...evaluationllmcapabilitythreshold+1Source ↗ demonstrates automated A/B testing improves persuasion by 35%

Medium-Term Risks (2026-2029)

Bioweapons Development

Critical Bottlenecks:

• Specialized synthesis knowledge for dangerous compounds
• Autonomous troubleshooting of complex laboratory procedures
• Multi-week experimental planning and adaptation
• Integration of theoretical knowledge with practical constraints

Expert Assessment: RAND Corporation (2024)↗🔗 web★★★★☆RAND CorporationRAND Corporation studyRAND research reports on AI and bioweapons risk are directly relevant to frontier AI evaluation policy, particularly debates around capability thresholds used in safety frameworks like Anthropic's RSP or OpenAI's preparedness framework.This RAND Corporation research report examines the risk of AI systems providing meaningful uplift to actors seeking to develop biological weapons, focusing on how to assess capa...existential-riskevaluationred-teamingcapabilities+6Source ↗ estimates 60% probability of crossing threshold by 2028.

Economic Displacement Thresholds

McKinsey's research↗🔗 web★★★☆☆McKinsey & CompanyAgents, Robots, and Us: Skill Partnerships in the Age of AI (McKinsey MGI)Content was inaccessible due to server restrictions; metadata is inferred from the URL, truncated title, and McKinsey MGI's known research focus. Treat specific statistics (e.g., '57%') with caution until verified from the primary source.A McKinsey Global Institute report examining how AI agents and robotics are reshaping labor markets and workforce skills. The report reportedly finds that 57% of workers may nee...labor-marketsautomationdeploymentgovernance+3Source ↗ indicates that current technologies could automate about 57% of U.S. work hours in theory. By 2030, approximately 27% of current work hours in Europe and 30% in the United States could be automated. Workers in lower-wage jobs are up to 14 times more likely to need to change occupations than those in highest-wage positions.

Coding Benchmark Update: The International AI Safety Report (October 2025)↗🔗 webInternational AI Safety Report (October 2025)This is an official interim update to the landmark International AI Safety Report, a major intergovernmental reference document on AI safety risks; highly relevant for those tracking global AI governance and the evolving capabilities landscape as of late 2025.A focused interim update to the International AI Safety Report, chaired by Yoshua Bengio, covering significant developments in AI capabilities and their risk implications betwee...ai-safetygovernancecapabilitiespolicy+4Source ↗ notes that coding capabilities have advanced particularly quickly. Top models now solve over 60% of problems in SWE-bench Verified, up from 40% in late 2024 and almost 0% at the beginning of 2024. However, Scale AI's SWE-Bench Pro↗🔗 webScale AI's SWE-Bench ProUseful reference for tracking AI coding capabilities and benchmark integrity; relevant to researchers monitoring AI progress thresholds in software engineering as a leading indicator of more autonomous AI systems.Scale AI introduces SWE-Bench Pro, an enhanced version of the SWE-Bench coding benchmark designed to address limitations in existing evaluations of AI software engineering capab...capabilitiesevaluationbenchmarkstechnical-safety+2Source ↗ shows a significant performance drop: even the best models (GPT-5, Claude Opus 4.1) score only 23% on harder, more realistic tasks.

Long-Term Control Risks (2027-2035+)

Strategic Deception (Scheming)

Key Uncertainties:

• Whether sophisticated strategic modeling can emerge from current training approaches
• Detectability of strategic deception capabilities during evaluation
• Minimum capability level required for effective scheming

Research Evidence:

• Anthropic Constitutional AI↗📄 paper★★★☆☆arXivConstitutional AI: Harmlessness from AI FeedbackConstitutional AI paper presenting a method for training AI systems to be harmless using AI feedback based on a set of constitutional principles, addressing a fundamental challenge in AI alignment and safety.Yanuo Zhou (2025)2,673 citationsanthropickb-sourceSource ↗ shows limited success in detecting deceptive behavior
• Redwood Research↗🔗 webRedwood Research: AI ControlRedwood Research is one of the leading technical AI safety organizations; their AI control framework and alignment faking research are frequently cited in both academic and policy discussions on managing risks from advanced AI systems.Redwood Research is a nonprofit AI safety organization that pioneered the 'AI control' research agenda, focusing on preventing intentional subversion by misaligned AI systems. T...ai-safetyalignmenttechnical-safetyred-teaming+5Source ↗ adversarial training reveals capabilities often hidden during evaluation

Current State & Trajectory

Capability Progress Rates

According to Epoch AI's analysis↗🔗 web★★★★☆Epoch AITraining Compute of Frontier AI Models Grows by 4-5x Per Year (Epoch AI)This Epoch AI analysis is frequently cited in AI governance and safety contexts to justify compute-based thresholds in policy proposals, such as executive orders requiring reporting above certain FLOP counts.Epoch AI analyzes historical trends in the training compute used for frontier AI models, finding that compute has grown approximately 4-5x per year. This rapid scaling has signi...computecapabilitiesai-safetygovernance+3Source ↗, training compute for frontier models grows 4-5x yearly. Their Epoch Capabilities Index shows frontier model improvement nearly doubled in 2024. METR's research↗🔗 web★★★★☆METRMETR Capability Evaluations Update: Claude Sonnet and OpenAI o1This METR update is directly relevant to responsible scaling policy debates, providing empirical capability assessments used by labs to determine whether frontier models cross safety-relevant autonomy thresholds before deployment.METR presents updated capability evaluations for Claude Sonnet and OpenAI o1 models, assessing whether these frontier AI systems approach autonomy thresholds relevant to safety ...evaluationcapabilitiesred-teamingai-safety+3Source ↗ shows AI performance on task length has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months.

Data Sources: Epoch AI capability tracking↗🔗 web★★★★☆Epoch AIEpoch AI - AI Research and Forecasting OrganizationEpoch AI is a key reference organization for empirical data on AI scaling trends; their compute and training run databases are widely cited in AI safety and governance discussions.Epoch AI is a research organization focused on investigating and forecasting trends in artificial intelligence, particularly around compute, training data, and algorithmic progr...capabilitiescomputegovernancepolicy+4Source ↗, industry benchmark results, expert elicitation.

Compute Scaling Projections

Leading Organizations

Key Uncertainties & Research Cruxes

Measurement Validity

The Berkeley CLTC Working Paper on Intolerable Risk Thresholds↗🔗 webBerkeley CLTC Working Paper on Intolerable Risk ThresholdsPublished November 2024 by Berkeley's CLTC, this working paper is relevant to policymakers and safety researchers working on AI red lines, model evaluations, and governance frameworks for frontier AI systems.This Berkeley Center for Long-Term Cybersecurity working paper examines how to define and operationalize 'intolerable risk' thresholds for AI systems, providing a framework for ...ai-safetygovernancepolicyevaluation+5Source ↗ notes that models effectively more capable than the latest tested model (4x or more in Effective Compute or 6 months worth of fine-tuning) require comprehensive assessment including threat model mapping, empirical capability tests, elicitation testing without safety mechanisms, and likelihood forecasting.

An interdisciplinary review of AI evaluation↗📄 paper★★★☆☆arXivinterdisciplinary review of AI evaluationInterdisciplinary review examining how AI benchmarks are used to evaluate model performance, capabilities, and safety, with attention to their growing influence on AI development and regulatory frameworks and concerns about evaluating high-impact capabilities.Maria Eriksson, Erasmo Purificato, Arman Noroozian et al. (2025)This interdisciplinary meta-review of ~100 studies examines critical shortcomings in quantitative AI benchmarking practices over the past decade. The paper identifies fine-grain...evaluationSource ↗ highlights the "benchmark lottery" problem: researchers at Google's Brain Team found that many factors other than fundamental algorithmic superiority may lead to a method being perceived as superior. Ironically, a majority of influential benchmarks have been released without rigorous peer review.

Threshold Sharpness

Sharp Threshold Evidence:

• Authentication systems↗📄 paper★★★☆☆arXivAuthentication systemsThis arxiv preprint demonstrates curriculum learning applied to multimodal deep learning for post-disaster analytics, relevant to AI safety through its focus on improving model robustness and reliability in high-stakes emergency response scenarios.Huseyin Fuat Alsan, Taner Arsan (2023)This paper proposes a curriculum learning approach for post-disaster analytics using multimodal deep learning models that jointly process images and text. The authors introduce ...capabilitiestrainingeconomiccapability+1Source ↗: Detection accuracy drops from 95% to 15% once generation quality crosses threshold
• Economic viability: McKinsey automation analysis↗🔗 web★★★☆☆McKinsey & CompanyMcKinsey State of AI in 2024Useful as an industry benchmark for understanding real-world AI adoption trends; content inaccessible at time of indexing due to access restrictions, so details are inferred from the report's known structure and prior editions.McKinsey's annual survey-based report tracking enterprise AI adoption, investment trends, and emerging risks across industries. The report provides quantitative benchmarks on ho...capabilitiesgovernancedeploymentevaluation+1Source ↗ shows 10-20% capability improvements create 50-80% cost advantage in many tasks
• Security vulnerabilities: Most exploits require complete capability to work at all

Gradual Scaling Evidence:

• Job displacement: Different tasks within roles automate at different rates
• Persuasion effectiveness: Incremental improvements in messaging quality yield incremental persuasion gains
• Domain expertise: Knowledge accumulation appears continuous rather than threshold-based

Strategic Deception Detection

Critical unsolved problems in capability assessment:

Timeline Projections & Warning Indicators

2025 Critical Thresholds

High Probability (70%+):

• Authentication Collapse: Real-time deepfakes become commercially viable
• Content Generation Saturation: Human-level quality across all text/image modalities

Medium Probability (40-70%):

• Code Automation: 50%+ of software engineering tasks automated
• Basic Situational Awareness: Systems understand evaluation vs. deployment contexts

2026-2027 Medium Probability Events

Early Warning System

Red Flag Indicators:

• Sudden benchmark improvements >20 percentage points
• Systems developing capabilities not explicitly trained for
• Gap between capability and safety evaluation results widening
• Evidence of strategic behavior during evaluation

Monitoring Infrastructure:

• METR↗🔗 web★★★★☆METRMETR: Model Evaluation and Threat ResearchMETR is a leading third-party AI safety evaluation organization whose work on autonomous capability benchmarks and catastrophic risk assessments directly informs AI lab safety policies and government AI governance frameworks.METR is an organization conducting research and evaluations to assess the capabilities and risks of frontier AI systems, focusing on autonomous task completion, AI self-improvem...evaluationred-teamingcapabilitiesai-safety+5Source ↗ dangerous capability evaluations
• MIRI↗🔗 web★★★☆☆MIRIMachine Intelligence Research InstituteMIRI is a foundational organization in the AI safety ecosystem; its research agenda and publications have significantly shaped the field's early theoretical frameworks.MIRI is a nonprofit research organization focused on ensuring that advanced AI systems are safe and beneficial. It conducts technical research on the mathematical foundations of...ai-safetyalignmentexistential-risktechnical-safety+2Source ↗ alignment evaluation protocols
• Industry responsible scaling policies (OpenAI Preparedness↗🔗 web★★★★☆OpenAIOpenAI Preparedness FrameworkThis is OpenAI's official Preparedness Framework page, relevant to discussions of frontier AI governance, deployment safety standards, and how leading labs operationalize risk thresholds before releasing powerful models.OpenAI's Preparedness initiative outlines a framework for tracking, evaluating, and mitigating catastrophic risks from frontier AI models. It establishes risk thresholds across ...ai-safetygovernanceevaluationred-teaming+5Source ↗, Anthropic RSP↗🔗 web★★★★☆AnthropicResponsible Scaling PolicyThis is Anthropic's foundational policy document establishing how it gates deployment of increasingly capable models; a key reference for understanding industry-led AI governance frameworks and voluntary safety commitments.Anthropic introduces its Responsible Scaling Policy (RSP), a framework of technical and organizational protocols for managing catastrophic risks as AI systems become more capabl...governancepolicyai-safetycapabilities+6Source ↗)
• Academic capability forecasting (Epoch AI↗🔗 web★★★★☆Epoch AIEpoch AI - AI Research and Forecasting OrganizationEpoch AI is a key reference organization for empirical data on AI scaling trends; their compute and training run databases are widely cited in AI safety and governance discussions.Epoch AI is a research organization focused on investigating and forecasting trends in artificial intelligence, particularly around compute, training data, and algorithmic progr...capabilitiescomputegovernancepolicy+4Source ↗)

The METR Common Elements Report (December 2025)↗🔗 web★★★★☆METRMETR's analysis of 12 companiesPublished by METR (Model Evaluation and Threat Research), this comparative analysis is useful for those tracking industry self-governance and responsible scaling policy developments across major AI labs.METR analyzes the safety policies of 12 frontier AI companies to identify common elements, commitments, and gaps in how organizations approach responsible deployment of advanced...ai-safetygovernancepolicyevaluation+6Source ↗ describes how each major AI developer's policy uses capability thresholds for biological weapons development, cyberattacks, autonomous replication, and automated AI R&D, with commitments to conduct model evaluations assessing whether models are approaching thresholds that could enable severe harm.

Expert Survey Findings

An OECD-affiliated survey on AI thresholds↗🔗 webOECD-affiliated survey on AI thresholdsPublished via Oxford's AIGI and affiliated with the OECD, this survey is directly relevant to ongoing international efforts (e.g., AI Safety Institutes, EU AI Act) to establish capability thresholds that trigger regulatory scrutiny of frontier AI models.This OECD-affiliated survey examines how thresholds and capability benchmarks should be defined and applied to advanced AI systems for governance and risk management purposes. I...governancecapabilitiesevaluationpolicy+5Source ↗ found that experts agreed if training compute thresholds are exceeded, AI companies should:

• Conduct additional risk assessments (e.g., via model evaluations)
• Notify an independent public body (e.g., EU AI Office, FTC, or AI Safety Institute)
• Notify the government

Participants noted that risk assessment frameworks from safety-critical industries (nuclear, maritime, aviation, healthcare, finance, space) provide valuable precedent for AI governance.

Sources & Resources

Primary Research

Government & Policy

Technical Benchmarks & Evaluation

Risk Assessment Research

Additional Sources