Risk Activation Timeline Model

An Anthropic article examining the core difficulties in assessing AI system capabilities and safety properties. It explores why robust evaluations are critical yet methodologically challenging, addressing gaps between benchmark performance and real-world behavior as well as the limitations of current evaluation frameworks.

This NTI analysis examines active screening approaches to prevent catastrophic bioweapons threats, focusing on detection and interdiction strategies. It addresses biosecurity governance frameworks and the technical and policy measures needed to reduce the risk of biological weapons development and use. The piece contributes to understanding how proactive monitoring can serve as a layer of defense against existential-level biological risks.

This page outlines the European Commission's comprehensive policy framework for AI, centered on promoting trustworthy, human-centric AI through the AI Act, AI Continent Action Plan, and Apply AI Strategy. It aims to balance Europe's global AI competitiveness with safety, fundamental rights, and democratic values. Key initiatives include AI Factories, the InvestAI Facility, GenAI4EU, and the Apply AI Alliance.

The Deepfake Detection Challenge (DFDC) is an initiative to advance research on detecting AI-generated synthetic media (deepfakes). It highlights the gap between rapidly improving deepfake generation capabilities and the slower development of reliable detection tools, reflecting a broader pattern of defensive research lagging behind offensive AI capabilities.

This URL was intended to link to Metaculus's collection of AI-related forecasting questions, a platform where forecasters make probabilistic predictions about AI development timelines, capabilities, and risks. However, the page currently returns a 404 error, indicating the content may have moved or been reorganized.

The official UK government page for the AI Safety Summit 2023, held November 1-2 at Bletchley Park, which convened governments, AI companies, civil society, and researchers to address frontier AI risks. Key outputs include the Bletchley Declaration—a multilateral agreement on AI safety—company safety policies, and a frontier AI capabilities and risks discussion paper. The summit marked a landmark moment in international AI governance coordination.

The 2022 ESPAI surveyed 738 machine learning researchers (NeurIPS/ICML authors) about AI progress timelines and risks, serving as a replication and update of the 2016 survey. Key findings include an aggregate forecast of 50% chance of HLMI by 2059 (37 years from 2022), with significant disagreement among experts about timelines and risks.

OpenAI's safety hub outlines their multi-stage approach to AI safety through teaching (value alignment and content filtering), testing (red teaming and preparedness evaluations), and sharing (real-world feedback loops). It covers key concern areas including child safety, deepfakes, bias, and election integrity, and links to their Preparedness Framework and related safety documentation.

METR is an organization conducting research and evaluations to assess the capabilities and risks of frontier AI systems, focusing on autonomous task completion, AI self-improvement risks, and evaluation integrity. They have developed the 'Time Horizon' metric measuring how long AI agents can autonomously complete software tasks, showing exponential growth over recent years. They work with major AI labs including OpenAI, Anthropic, and Amazon to evaluate catastrophic risk potential.

This paper addresses the critical role of model evaluation in mitigating extreme risks from advanced AI systems. As AI development progresses, general-purpose AI systems increasingly possess both beneficial and harmful capabilities, including potentially dangerous ones like offensive cyber abilities or manipulation skills. The authors argue that two types of evaluations are essential: dangerous capability evaluations to identify harmful capacities, and alignment evaluations to assess whether models are inclined to use their capabilities for harm. These evaluations are vital for informing policymakers and stakeholders, and for making responsible decisions regarding model training, deployment, and security.

IBM's annual Cost of a Data Breach Report, produced with the Ponemon Institute, provides global research on data breach costs, trends, and contributing factors. The 2025 edition highlights an 'AI oversight gap' where rapid AI adoption is outpacing security governance, with ungoverned AI systems facing higher breach likelihood and costs. The global average breach cost reached $4.4M USD.

This paper presents DeepSeek LLM, an open-source large language model project that addresses inconsistencies in scaling law literature by providing empirical findings for scaling models at 7B and 67B parameters. The authors developed a 2 trillion token dataset and applied supervised fine-tuning and Direct Preference Optimization to create DeepSeek Chat models. Their evaluation demonstrates that DeepSeek LLM 67B outperforms LLaMA-2 70B across multiple benchmarks, particularly in code, mathematics, and reasoning tasks, with the chat variant showing competitive performance against GPT-3.5.

The Open LLM Leaderboard is a HuggingFace-hosted benchmarking platform that compares open-source large language models across standardized evaluations in a transparent and reproducible manner. It allows researchers and practitioners to filter, search, and rank models by performance metrics, providing a community reference for tracking AI capabilities progress. The leaderboard has since been archived, reflecting the rapid pace of LLM development.

The NIST AI RMF is a voluntary, consensus-driven framework released in January 2023 to help organizations identify, assess, and manage risks associated with AI systems while promoting trustworthiness across design, development, deployment, and evaluation. It provides structured guidance organized around core functions and is accompanied by a Playbook, Roadmap, and a Generative AI Profile (2024) addressing risks specific to generative AI systems.

The FTC warns consumers about AI-powered voice cloning scams where fraudsters impersonate distressed family members to extract money. The post explains how scammers use readily available AI tools to clone voices from social media and other public audio, and offers guidance on how to verify calls and protect against such fraud.

The Center for AI Safety (CAIS) is a research organization focused on mitigating catastrophic and existential risks from advanced AI systems. It conducts technical research, publishes surveys and statements, and supports field-building efforts across academia and industry. CAIS is notable for its broad coalition-building, including its widely-cited statement on AI extinction risk signed by leading researchers.

Reuters analysis examining how AI-generated misinformation poses risks to the 2024 election cycle, covering deepfakes, synthetic media, and coordinated disinformation campaigns. The piece assesses the scale of the threat and the challenges platforms and regulators face in detection and mitigation.

RAND Corporation's AI research hub covers policy, national security, and governance implications of artificial intelligence. It aggregates reports, analyses, and commentary on AI risks, military applications, and regulatory frameworks from one of the leading U.S. defense and policy think tanks.

Epoch AI's analysis of historical trends in compute used for training notable AI systems, identifying three distinct eras: pre-deep learning, deep learning, and large-scale models. The research documents how training compute has grown by roughly 4-5 orders of magnitude since 2010, with a notable shift toward massive compute investment after 2015.

This resource from the Future of Humanity Institute (FHI) at Oxford involves expert elicitation surveys focused on AI development timelines, capability thresholds, and prioritization of interventions. It aggregates forecasts from researchers to inform understanding of when transformative AI might arrive and what safety measures may be most effective.

This Stanford HAI resource examines the challenges AI tools—particularly large language models—pose to academic integrity, exploring how institutions should respond to AI-assisted cheating and the broader implications for education and assessment. It likely draws on Stanford research to assess the prevalence and nature of AI misuse in academic settings and proposes policy frameworks for educators.

A CSET analysis examining how artificial intelligence is reshaping the cybersecurity landscape, covering both offensive and defensive applications of AI in cyber operations. The report assesses near- and long-term implications of AI capabilities for cyber threats, defenses, and policy responses.

The C2PA is an industry coalition that has developed an open technical standard for attaching verifiable provenance metadata to digital content, functioning like a 'nutrition label' that tracks a file's origin, creation tools, and edit history. This standard aims to help consumers and platforms distinguish authentic content from manipulated or AI-generated media. It is backed by major technology and media companies including Adobe, Microsoft, and the BBC.