Bioweapons

Quick Assessment

Overview

AI systems could accelerate biological weapons development by helping with pathogen design, synthesis planning, or acquisition of dangerous knowledge. The concern isn't that AI creates entirely new risks, but that it lowers barriers—making capabilities previously requiring rare expertise more accessible to bad actors.

This is considered one of the most severe near-term AI risks because biological weapons can cause mass casualties and AI-assisted bioweapons could be developed by smaller groups than traditional state programs required. Unlike many other AI risks that depend on future, more capable systems, this risk applies to models available today.

The key debate centers on whether AI provides meaningful "uplift"—whether it genuinely helps beyond what's already accessible through scientific literature and internet searches, or whether wet-lab skills remain the true bottleneck. Current evidence is reportedly mixed: a 2024 RAND Corporation study found no statistically significant AI uplift for bioweapon attack planning,[^1] while separate Microsoft research indicated that AI-designed toxins evaded more than 75% of SecureDNAOrganizationSecureDNAA Swiss nonprofit foundation providing free, privacy-preserving DNA synthesis screening software using novel cryptographic protocols.Quality: 60/100 tools.[^2]

However, 2025 has marked a significant shift in official assessments. OpenAI has stated it expects its next-generation models to reach "high-risk classification" for biological capabilities—meaning they could provide "meaningful counterfactual assistance to novice actors."[^3] Anthropic reportedly activated ASL-3 (AI Safety Level 3) protections for Claude Opus 4 specifically due to biological and chemical weapon concerns.[^4] The National Academies of Sciences, Engineering, and Medicine's March 2025 report The Age of AI in the Life Sciences found that while current biological design tools cannot yet design self-replicating pathogens, monitoring and mitigation are urgently needed.[^5] OpenAI's o3 model has also scored at the 94th percentile on the Virology Capabilities Test (VCT), a benchmark designed to measure dangerous biological knowledge.[^6]

Risk Assessment

Responses That Address This Risk

The Total Risk Debate

How dangerous is AI-assisted bioweapons development? Expert assessments vary substantially, from those who consider it an imminent catastrophic threat to those who view it as overhyped. Understanding both sides of this debate—and the key uncertainties that drive disagreement—is essential for calibrating policy responses.

Estimating Overall Risk

Attempting to quantify the total risk from AI-assisted bioweapons requires estimating both the probability of an attack and its potential consequences. Estimates vary widely:

Expert Survey Data (2025)

The Forecasting Research Institute surveyed 46 biosecurity experts and 22 superforecasters in early 2025 on AI-enabled biorisk.[^7] Their findings are summarized below:[^8]

Key insight: According to the FRI survey, safeguards (closed weights, anti-jailbreaking, DNA screening) can reduce risk nearly to baseline even with advanced AI capabilities.[^9]

The Bioweapons Attack Chain ModelAnalysisBioweapons Attack Chain ModelMultiplicative attack chain model estimates catastrophic bioweapons probability at 0.02-3.6%, with state actors (3.0%) showing highest estimated risk due to lab access. DNA synthesis screening offe...Quality: 69/100 estimates compound attack probability at 0.02%–3.6% depending on assumptions, with substantial uncertainty at each step. The wide range reflects genuine disagreement about key parameters.

Existential risk context: In The Precipice (2020), Oxford philosopher Toby OrdPersonToby OrdComprehensive biographical profile of Toby Ord documenting his 10% AI extinction estimate and role founding effective altruism, with detailed tables on risk assessments, academic background, and in...Quality: 41/100 estimates the chance of existential catastrophe from engineered pandemics at 1 in 30 by 2100—which he identifies as second only to AI among anthropogenic risks.[^10] Ord writes that it "now seems within the reach of near-term biological advances to create pandemics that would kill greater than 50% of the population—not just in a particular area, but globally."[^11] While not all engineered pandemics would be AI-assisted, this frames the potential severity of the threat.

Industry concerns: In July 2023, Anthropic CEO Dario AmodeiPersonDario AmodeiComprehensive biographical profile of Anthropic CEO Dario Amodei documenting his competitive safety development philosophy, 10-25% catastrophic risk estimate, 2026-2030 AGI timeline, and Constituti...Quality: 41/100 stated that within two to three years, there was a "substantial risk" that AI tools would "greatly widen the range of actors with the technical capability to conduct a large-scale biological attack."[^12] The Center for a New American Security (CNAS) report on AI and bioweapons notes this could "expose the United States to catastrophic threats far exceeding the impact of COVID-19."[^13]

Arguments for High Risk

Those who consider AI-bioweapons a severe threat emphasize several points:

1. Democratization of Dangerous Knowledge

AI makes dangerous biological knowledge more accessible to those who couldn't previously obtain it. While scientific literature contains detailed protocols, navigating it requires expertise. AI systems can synthesize, explain, and contextualize this information for non-experts, potentially expanding the pool of capable actors.

The equalizer effect: The most concerning scenario isn't AI helping expert virologists (who already have the knowledge), but AI helping moderately skilled individuals bridge knowledge gaps that previously required years of training or team collaboration.

2. Asymmetric Evasion Capabilities

According to reporting on Microsoft's 2024 research, AI-designed toxins reportedly evaded a substantial proportion of commercial DNA synthesis screening tools.[^2] This is qualitatively different from knowledge provision—it represents AI helping attackers circumvent existing defenses.

DNA synthesis screening is a cornerstone of current biosecurity. If AI can reliably design functional variants that evade detection, the entire screening paradigm may become obsolete faster than new defenses can be developed. This creates an asymmetric threat where even modest AI capabilities could undermine years of defensive investment.

3. Rapid Capability Improvement

AI capabilities are improving rapidly. Even if current models provide limited uplift, the trend is concerning:

2025 milestone: OpenAI's April 2025 o3 model reportedly ranked in the 94th percentile among expert human virologists on virology capability evaluations, marking the first time a frontier AI model has demonstrated expert-level performance on biological troubleshooting scenarios.[^14]

The argument is that we should prepare for future capabilities, not just current ones. By the time AI demonstrably provides high uplift, it may be too late to establish governance.

4. Combination with Other Technologies

AI alone may provide limited uplift, but the combination of multiple technologies could be transformative:

flowchart TD
  LLM[Large Language Models] --> COMBO[Compound Capability]
  PROTEIN[Protein Design AI] --> COMBO
  LAB[Lab Automation] --> COMBO
  SYNTH[Cheap DNA Synthesis] --> COMBO
  COMBO --> THREAT[Enhanced Threat]

  style COMBO fill:#ffddcc
  style THREAT fill:#ffcccc

• LLMs + protein design tools: Tools such as AlphaFold, which DeepMind released publicly in 2021, enable novel protein structure prediction and engineering; LLMs help identify targets and plan experimental applications.[^15]
• AI + lab automation: Automated systems could eventually execute protocols with minimal human intervention
• AI + decreasing synthesis costs: DNA synthesis costs have fallen dramatically over the past two decades; AI could help design sequences optimized for synthesis on cheaper platforms.[^16]

Each technology alone may be manageable, but their combination could create emergent risks that exceed any individual contribution.

5. Tail Risk Considerations

Even if the median expectation is manageable, the worst-case scenarios are severe enough to warrant serious attention:

• Engineered pandemic: A pathogen designed for transmissibility, lethality, and immune evasion could potentially cause millions of deaths
• Multiple simultaneous attacks: AI could enable coordination of attacks across multiple locations
• Degradation of trust in biology: Widespread bioterrorism could undermine beneficial biological research and public health

From a risk management perspective, low-probability/high-consequence events may deserve more weight than their expected value alone suggests.

6. Historical Underestimation

History suggests we systematically underestimate technology-enabled threats. The first nuclear device was tested in July 1945—less than a decade after the discovery of fission in 1938, a pace faster than many contemporary physicists anticipated.[^17] COVID-19 demonstrated how disruptive a novel pathogen can be, causing millions of deaths and trillions of dollars in economic damage within months of its emergence.[^18] AI capabilities have also repeatedly exceeded near-term forecasts.

Skepticism about AI-bioweapons risk may itself be the risky position.

7. The "De-skilling" Trajectory

Multiple emerging technologies are simultaneously reducing the skill requirements for biological research:

• Cloud laboratories automate complex procedures and allow remote execution
• Benchtop DNA synthesizers are approaching gene-length capabilities
• AI assistants bridge knowledge gaps and provide troubleshooting guidance
• Protocol automation reduces the need for tacit laboratory knowledge

Each of these alone might be manageable, but together they suggest a trajectory toward dramatically lowered barriers. Any current empirical study may capture a snapshot where these technologies haven't yet converged—but convergence appears plausible within the decade.

8. Offense Has Asymmetric Advantages

Biological attacks have inherent asymmetric characteristics that favor attackers:

• Attribution lag: Days to weeks may pass before an attack is recognized as intentional
• Preparation asymmetry: Attackers can prepare countermeasures for themselves; defenders must protect everyone
• Innovation asymmetry: Attackers need to succeed once; defenders must anticipate all possible attack vectors
• Psychological impact: Even unsuccessful or small-scale attacks could cause massive economic and social disruption

AI amplifies these asymmetries by potentially enabling novel attack vectors that existing defenses haven't anticipated.

9. Open-Source Model Proliferation

Even if frontier labs implement strong biosecurity measures, the proliferation of open-source models undermines containment:

• No centralized control: Once weights are released, restrictions cannot be enforced
• Fine-tuning vulnerability: Safety training can be removed with relatively modest compute
• Capability improvements: Open models are approaching frontier capabilities with roughly 6–12 month lags
• Global availability: Actors in any jurisdiction can access open models

The CNAS report↗🔗 web★★★★☆CNASAI and the Evolution of Biological National Security RisksA CNAS policy report providing a broad overview of AI-biosecurity intersection for policymakers; useful for understanding governance challenges around dual-use AI capabilities in the biological domain.This CNAS report examines how AI advancements intersect with biosecurity risks, analyzing threats from state actors, nonstate actors, and accidental releases. It assesses whethe...biosecuritydual-use-researchexistential-riskgovernance+5Source ↗ recommends considering a "licensing regime for biological design tools with potentially catastrophic capabilities"—but this has not been implemented as of 2025.[^19]

The DeepSeek warning: In February 2025, Anthropic CEO Dario AmodeiPersonDario AmodeiComprehensive biographical profile of Anthropic CEO Dario Amodei documenting his competitive safety development philosophy, 10-25% catastrophic risk estimate, 2026-2030 AGI timeline, and Constituti...Quality: 41/100 reportedly stated that testing of China's DeepSeek model revealed it was among the worst performers on biosecurity of any model evaluated—generating information relevant to producing bioweapons "that can't be found on Google or can't be easily found in textbooks" with "absolutely no blocks whatsoever."[^20] While Amodei did not characterize DeepSeek as "literally dangerous" at that time, the incident highlighted how open-source models from different jurisdictions may not implement equivalent safety measures.[^21]

Arguments for Lower Risk

Those who consider AI-bioweapons risk overstated emphasize different considerations:

1. The RAND Study: No Significant Uplift

A 2024 RAND Corporation study is among the more rigorous empirical assessments of AI uplift conducted to date. According to reporting on the study, twelve teams of three researchers each spent 80 hours developing bioweapon attack plans—half using AI assistance, half using only open internet resources. Expert evaluators reportedly found no statistically significant difference in plan viability between the two groups.[^1]

This finding directly challenges claims that AI meaningfully assists biological attacks. If AI-assisted and non-AI teams perform equivalently, the AI "threat" may be more limited than feared.

Implications: Dangerous biological information is already widely accessible through legitimate scientific literature. AI may be redundant with existing sources rather than providing novel dangerous capabilities.

2. Wet Lab Bottleneck

Knowledge is not capability. Even with complete theoretical understanding, executing biological synthesis requires:

• Tacit knowledge that transfers poorly through text (how to handle contamination, optimize growth conditions, troubleshoot failures)
• Specialized equipment that is expensive, regulated, and hard to obtain
• Months of practice to develop reliable technique
• Physical safety procedures that untrained individuals typically violate

The Soviet Union's Biopreparat program, established in the 1970s, reportedly employed tens of thousands of scientists and technicians over decades in a state-directed effort to develop reliable bioweapons—a scale of human expertise that underscores the difficulty of the task.[^22] Aum Shinrikyo, despite substantial financial resources and personnel with scientific training, failed repeatedly in their bioweapons attempts throughout the 1990s.[^23] The capability bottleneck may be far more important than the knowledge bottleneck.

AI cannot transfer tacit knowledge. Reading about sterile technique is different from maintaining it reliably under pressure. AI can explain protocols but cannot teach hands-on laboratory skills.

3. Guardrails and Filtering Work

Frontier AI models include safety measures that reduce dangerous information provision:

• Refusals for explicitly harmful requests
• Content filtering
• Constitutional AI and RLHF training
• Continuous red-teaming and patching

While not perfect, these measures raise barriers. Jailbreaking techniques exist but require effort and sophistication, and often produce degraded responses. The marginal attacker may be more likely to use open internet resources than to navigate AI guardrails.

4. Existing Information Abundance

Scientific literature already contains dangerous information. Textbooks explain pathogen biology in detail. The internet hosts synthesis protocols. The marginal information contribution of AI may be minimal when the baseline is that much of this information is already accessible. AI's value proposition—synthesis and accessibility—matters less if motivated individuals were already able to find information through traditional means.

5. Defense Advantages

AI capabilities benefit defense as much as offense, and defensive applications may be more scalable:

Metagenomic surveillance, mRNA vaccine platforms, and AI-assisted drug discovery are advancing rapidly. These defensive technologies may ultimately make biological attacks less effective rather than more dangerous.

The transition period concern: Even those who believe defense wins long-term often worry about a near-term window where offense temporarily gains advantages before defenses mature.

6. Deterrence and Attribution

Biological attacks, especially sophisticated ones, leave traces that can enable attribution:

• Genomic sequencing of pathogens
• Epidemiological tracking
• Intelligence on precursor purchases
• Surveillance of likely actors

State actors face retaliation risks. Non-state actors face intense investigative focus. The certainty of attribution for significant attacks provides a deterrent effect that pure capability analysis misses.

7. Historical Non-Occurrence

Despite decades of accessible biological knowledge and multiple motivated actors, catastrophic bioterrorism has not occurred. This may indicate genuine difficulty—or it may reflect luck that could change as AI lowers barriers.

The Key Cruxes

Much of the disagreement about AI-bioweapons risk reduces to a small number of factual questions where reasonable people disagree:

Crux 1: Does AI Provide Meaningful Uplift?

If uplift is low (less than 1.5x): Focus resources on traditional biosecurity rather than AI-specific interventions. The threat is real but not qualitatively changed by AI.

If uplift is high (greater than 2x): Urgent need for AI-specific guardrails, compute governance, and model restrictions. The threat landscape has fundamentally shifted.

Crux 2: Is the Knowledge Bottleneck or Capability Bottleneck More Important?

If knowledge is the bottleneck: AI providing information is directly dangerous.

If capability is the bottleneck: AI providing information is mostly redundant with existing sources; wet lab skills remain rate-limiting.

Crux 3: Will Defense or Offense Win Long-Term?

If defense wins: AI-bioweapons is a transitional problem that self-corrects as defensive applications mature.

If offense wins: AI permanently shifts the advantage to attackers, requiring sustained containment efforts.

If it's a window: The near-term favors offense, but defense catches up—the question is whether catastrophic attacks occur during the transition.

Crux 4: How Quickly Are Capabilities Advancing?

If capabilities are saturating: Current systems represent near-peak dangerous capabilities; governance can catch up.

If capabilities continue scaling: Future systems will be substantially more dangerous; governance is racing against time.

The AI-Bioweapons Timeline ModelAnalysisAI-Bioweapons Timeline ModelTimeline model projects AI-bioweapons capabilities crossing four thresholds: knowledge democratization already partially crossed (fully by 2025-2027), synthesis assistance 2027-2032 (median 2029), ...Quality: 58/100 projects capability thresholds, with synthesis assistance potentially arriving 2027-2032 and novel agent design 2030-2040.

Crux 5: How Effective Are Guardrails and Countermeasures?

If guardrails work well: The marginal risk from AI models is small; responsible development practices suffice.

If guardrails fail: Open-source proliferation and jailbreaking make model-level interventions largely ineffective.

The open-source challenge: Even if frontier labs implement strong safeguards, open-source models may not. As capable open models proliferate, guardrails become optional, fine-tuning can remove remaining restrictions, and dangerous capabilities become permanently accessible.

Crux 6: Can DNA Synthesis Screening Keep Pace?

DNA synthesis screening is the primary defense against engineered pathogens, but Microsoft's research revealed significant gaps.

If screening adapts: AI-designed evasion is a temporary problem; screening improvements restore the chokepoint.

If screening falls behind: The primary technical barrier erodes; other defenses must compensate.

Key questions:

• Can screening adapt to AI-designed evasive sequences?
• What happens as benchtop synthesis equipment becomes cheaper and more accessible?
• Can screening extend to cover novel synthesis methods and cloud laboratories?

The Framework for Nucleic Acid Synthesis Screening↗🏛️ governmentFramework for Nucleic Acid Synthesis ScreeningRelevant to AI safety discussions around dual-use technology governance; this framework models how governments can establish screening and safeguards for powerful biotechnologies, offering lessons for analogous AI governance challenges.The Biden White House Office of Science and Technology Policy (OSTP) released a framework establishing standards for screening nucleic acid synthesis orders to prevent misuse fo...biosecuritygovernancepolicyexistential-risk+4Source ↗ (April 2024) represents a policy response, but only applies to federally funded programs.

Current Evidence

Studies have shown Large Language ModelsCapabilityLarge Language ModelsComprehensive analysis of LLM capabilities showing rapid progress from GPT-2 (1.5B parameters, 2019) to GPT-5 and Gemini 2.5 (2025), with training costs growing 2.4x annually and projected to excee...Quality: 60/100 can provide information relevant to bioweapon development, though the significance is contested.

RAND Red-Team Study (2024)

The RAND Corporation study ("The Operational Risks of AI in Large-Scale Biological Attacks") is reportedly one of the more rigorous empirical assessments of AI uplift conducted to date.¹ Researchers Christopher Mouton, Caleb Lucas, and Ella Guest reportedly recruited 15 groups of three people to act as Red TeamingResearch AreaRed TeamingRed teaming is a systematic adversarial evaluation methodology for identifying AI vulnerabilities and dangerous capabilities before deployment, with effectiveness rates varying from 10-80% dependin...Quality: 65/100 "bad guys."¹

According to the study, twelve teams were given 80 hours each over seven weeks to develop bioweapon attack plans based on one of four scenarios—including a "fringe doomsday cult intent on global catastrophe" and a "private military company seeking to aid an adversary's conventional military operation."¹ For each scenario, one team had access to an Large Language ModelsConceptLarge Language ModelsComprehensive assessment of LLM capabilities showing training costs growing 2.4x/year ($78-191M for frontier models, though DeepSeek achieved near-parity at $6M), o3 reaching 91.6% on AIME and 87.5...Quality: 62/100 chatbot, another had a different chatbot, and control teams used only internet resources.¹

Expert judges (biologists and security specialists) evaluated the resulting plans for biological and operational feasibility. The reported result: no statistically significant difference in plan viability between AI-assisted and non-AI groups.¹

Key methodology details:

• Participants had some technical background (science graduates)
• Testing focused on planning, not actual synthesis
• Used 2023-era models; capabilities have advanced since
• Sample size was relatively small (n=12 teams completing the study)
• LLMs did not generate explicit weaponization instructions, but reportedly provided "guidance and context in critical areas such as agent selection, delivery methods, and operational planning"¹

Limitations acknowledged by researchers: The study tested planning capability, not execution. It used participants with technical backgrounds, so may underestimate uplift for complete novices. AI capabilities continue advancing.

Implications: The wet-lab bottleneck may be more significant than the knowledge bottleneck. Knowing how to make something is different from being able to make it.

AI-Designed Toxins Evade Screening (2024)

Microsoft researchers reportedly conducted a red-team exercise testing biosecurity in the protein engineering pipeline. According to some sources, DNA screening software—used by synthesis companies to flag dangerous sequences—missed over 75% of AI-designed potential toxins, with one tool flagging only 23% of sequences.² After the research was published, screening systems reportedly improved to catch approximately 72% on average.²

Key details:

• Tested multiple commercial screening tools
• AI reportedly designed functional variants that differed sufficiently from known threats to evade pattern matching
• Improvement after publication shows screening can adapt—but also shows it wasn't keeping pace

Implications: Even if current LLMs provide limited knowledge uplift, AI protein design tools may create harder-to-detect threats. The screening ecosystem has significant gaps that AI can exploit.

Gryphon Scientific Evaluation (2023)

Anthropic hired Gryphon Scientific to red-team Claude's ability to provide harmful biological information.³ According to reports, the evaluation involved more than 150 hours of testing and drew on more than 20 biosecurity experts.³

The findings were described as concerning. Rocco Casagrande, Gryphon's managing director, reportedly stated he was "personally surprised and dismayed by how capable current LLMs were at providing critical information related to biological weapons."³ He was quoted by Semafor as saying: "These things are developing extremely, extremely fast, they're a lot more capable than I thought they would be when it comes to science."³

Key findings (according to reports):

• One team member with a postdoctoral fellowship studying a pandemic-capable virus found LLMs could provide "post-doc level knowledge to troubleshoot commonly encountered problems" when working with that virus
• For low-skill users, LLMs could suggest which viruses to acquire
• Although LLMs often hallucinate, they answered almost all questions accurately at least sometimes, and answered some critical questions nearly always accurately
• Workshops with biosecurity experts identified concerning misuse scenarios including how to reconstruct information redacted from sensitive scientific documents

Despite the concerning findings, Casagrande reportedly believes "concerted action could ensure safety is built into the most advanced models."³

Anthropic, OpenAI Evaluations

AI labs have conducted extensive internal evaluations testing whether their models could provide "uplift" to potential bioweapon developers.

Anthropic's approach: Anthropic's Responsible Scaling PoliciesApproachResponsible Scaling PoliciesComprehensive analysis of Responsible Scaling Policies showing 20 companies with published frameworks as of Dec 2025, with SaferAI grading major policies 1.9-2.2/5 for specificity. Evidence suggest...Quality: 62/100 (RSP) defines AI Safety Levels (ASL) modeled after biosafety level (BSL) standards.⁴ They reportedly conduct at least 10 different biorisk evaluations for each major model release.⁴ In early 2025, Anthropic reportedly sent a letter to the White House urging immediate action on AI security after its testing revealed alarming improvements in Claude 3.7 Sonnet's ability to assist with aspects of bioweapons development.⁵

OpenAI's framework: OpenAI's Preparedness Framework categorizes biological and chemical capabilities as "Tracked Categories" requiring ongoing evaluation.⁶ They define two thresholds:

• High capability: Could "provide meaningful counterfactual assistance to 'novice' actors (anyone with a basic relevant technical background) that enables them to create known biological or chemical threats"⁶
• Critical capability: Could "introduce unprecedented new pathways to severe harm"⁶

OpenAI states their most advanced models "aren't yet capable enough to pose severe risks" in biosecurity—but has reportedly indicated upcoming models may reach "high" capability level.⁶

US/UK AI Safety InstituteOrganizationUK AI Safety InstituteThe UK AI Safety Institute (renamed AI Security Institute in Feb 2025) operates with ~30 technical staff and 50M GBP annual budget, conducting frontier model evaluations using its open-source Inspe...Quality: 52/100 joint evaluation (2024): The first joint government-led model evaluation tested Claude 3.5 Sonnet across biological capabilities, cyber capabilities, software development, and safeguard efficacy.⁷ Elizabeth KellyPersonElizabeth KellyFormer Director of the US AI Safety Institute (AISI), who led federal AI safety evaluation and policy efforts until her departure in February 2025., AISI director, was quoted as calling it "the most comprehensive government-led safety evaluation of an advanced AI model to date."⁷

Evaluation Methodology Limitations

An Epoch AI analysis of biorisk evaluations across major AI labs identified significant methodological concerns:[^31]

Key findings (according to Epoch AI):[^31]

• Most publicly described biorisk benchmarks have "rapidly saturated"—AI systems now exceed expert-human baselines
• Benchmarks "practically always fail to capture many real-world complexities"
• Anthropic is the only frontier lab conducting explicit biorisk uplift trials
• Despite limitations, Epoch AI concluded Anthropic was "largely justified" in activating ASL-3

Kevin Esvelt's Classroom Experiment

MIT researcher Kevin Esvelt reportedly conducted an informal demonstration in which he asked students to use ChatGPT or other LLMs to identify dangerous pathogens.[^32] According to some accounts, after approximately one hour, the class had identified four potential pandemic pathogens, methods to generate them from synthetic DNA, names of DNA synthesis companies unlikely to screen orders, and detailed protocols and troubleshooting guidance.[^32]

Esvelt was quoted regarding AI's ability to circumvent DNA screening defenses: "We've built a Maginot Line of defense, and AI just walked around it."[^32]

This demonstration, while not a rigorous study, illustrates how quickly accessible LLMs can be leveraged for potentially dangerous information-gathering—even for those without prior expertise.

CNAS Report: AI and Biological National Security Risks (2024)

The Council on Strategic RisksOrganizationCouncil on Strategic RisksThe Council on Strategic Risks is a DC-based nonprofit founded in 2017 that focuses on climate-security intersections, strategic weapons, and ecological risks through three research centers. While ...Quality: 38/100 report by Bill Drexel and Caleb Withers provides a comprehensive analysis of the evolving AI-biosecurity landscape.[^33]

Key concerns identified:[^33]

• AI could enable bioterrorism, create unprecedented superviruses, and develop novel targeted bioweapons
• AI's potential to "optimize bioweapons for targeted effects, such as pathogens tailored to specific genetic groups or geographies, could significantly shift states' incentives to use biological weapons"
• If realized, such threats could "expose the United States to catastrophic threats far exceeding the impact of COVID-19"

Key recommendations:[^33]

• Strengthen screening mechanisms for cloud labs and genetic synthesis providers
• Conduct rigorous assessments of foundation models' biological capabilities throughout the bioweapons lifecycle
• Invest in technical safety mechanisms to curb threats posed by foundation models
• Consider a licensing regime for biological design tools with potentially catastrophic capabilities

The report emphasizes that while AI-enabled biological catastrophes are "far from inevitable," current biological safeguards already need significant updates.[^33]

2025–2026 Developments: A Pivotal Period

2025 marked a significant shift in how AI labs and governments assess biological risks. According to the Council on Strategic Risks: "The year 2025 brought rising public awareness and discussion of the risks at the AI-biology nexus."[^34]

Several specific developments stand out:

OpenAI's High-Risk Classification

OpenAI reportedly announced that upcoming models—particularly successors to the o3 reasoning model—may trigger "high-risk classification" under its Preparedness Framework.[^35] This would mean they could provide "meaningful counterfactual assistance to novice actors" in creating known biological threats.[^35]

Key points from OpenAI's approach (according to reports):[^35]

• Classified ChatGPT Agent as having "High capability in the biological domain"
• Discovered that creating bioweapons would require weeks or months of sustained AI interaction, not single conversations
• Implemented a traffic-light system: red-level content (direct bioweapon assistance) is immediately blocked; yellow-level content (dual-use information) requires careful handling

Anthropic's ASL-3 Activation (May 2025)

Anthropic reportedly became the first lab to activate its highest safety tier (ASL-3) specifically for biological concerns when releasing Claude Opus 4.[^36] Their internal evaluations reportedly found they "could no longer confidently rule out the ability of our most advanced model to uplift people with basic STEM backgrounds" attempting to develop CBRN weapons.[^36]

Anthropic's testing reportedly revealed:[^36]

• Participants with access to Claude Opus 4 developed bioweapon acquisition plans with "substantially fewer critical failures" than internet-only controls
• Claude went from underperforming world-class virologists to "comfortably exceeding that baseline" on virology troubleshooting within a year

National Academies Report (March 2025)

The National Academies of Sciences, Engineering, and Medicine published "The Age of AI in the Life Sciences: Benefits and Biosecurity Considerations," reportedly directed by Executive Order 14110.[^37] Key findings included:[^37]

• AI-enabled biological tools can improve biosecurity through enhanced surveillance and faster countermeasure development
• Current biological design tools can design simpler structures (molecules) but cannot yet design self-replicating pathogens
• A "distinct lack of empirical data" exists for evaluating biosecurity risks of AI-enabled biological tools
• Recommended continued investment alongside monitoring for potential risks

CSIS Policy Analysis (August 2025)

The Center for Strategic and International Studies reportedly published "Opportunities to Strengthen U.S. Biosecurity from AI-Enabled Bioterrorism," warning that current U.S. biosecurity measures are "ill-equipped to meet these challenges."[^38] The report noted that critical safeguards in biological design tools are "already circumventable post-deployment."[^38]

Supplementary Evidence

How AI Could Help Attackers

AI could assist at multiple stages of bioweapon development:

Attack Chain Analysis

A successful biological attack requires success across multiple stages, each with independent failure modes:

flowchart TD
  A[Motivation] --> B[AI/Information Access]
  B --> C[Knowledge Uplift]
  C --> D[Lab Access]
  D --> E[Synthesis Success]
  E --> F[Deployment]
  F --> G[Evades Countermeasures]
  G --> H[Catastrophic Attack]

  style A fill:#fee
  style H fill:#fcc

See Bioweapons Attack Chain ModelAnalysisBioweapons Attack Chain ModelMultiplicative attack chain model estimates catastrophic bioweapons probability at 0.02-3.6%, with state actors (3.0%) showing highest estimated risk due to lab access. DNA synthesis screening offe...Quality: 69/100 for detailed probability estimates at each stage.

Specific Assistance Pathways

Target identification — AI might help identify dangerous modifications to known pathogens or find novel biological agents. Large language models trained on scientific literature have extensive knowledge of pathogen biology.

Synthesis planning — AI could help determine how to create dangerous biological materials. Protein design tools can generate novel sequences, and LLMs can explain synthesis routes.

Knowledge bridging — Most concerningly, AI might help bridge knowledge gaps. Historically, bioweapons development required rare combinations of expertise. AI could help a motivated individual or small group compensate for missing knowledge, potentially replacing what previously required teams of specialists.

Evasion optimization — AI could help design pathogens or synthesis routes that evade detection by screening tools, surveillance systems, or medical countermeasures.

History & Current Infrastructure

Biological threats exist on a spectrum. State programs have historically been the main concern, but the barrier to entry may be dropping. The COVID-19 pandemic demonstrated how much damage pathogens can cause and highlighted gaps in biosecurity infrastructure.

Historical Programs

State Bioweapons Programs

Multiple nations have maintained offensive biological weapons programs despite the Biological Weapons Convention (BWC):⁸

These programs required vast resources, thousands of scientists, and state-level infrastructure. The concern is that AI could reduce these requirements.

Current compliance concerns: According to some sources, the 2024 State Department report raised BWC compliance concerns about China, Russia, Iran, and North Korea.⁹ Verification remains difficult because the BWC has no formal verification regime.⁸

The Soviet Biopreparat Program: A Case Study

The Soviet Union reportedly operated one of the world's largest biological weapons programs—in direct violation of the BWC it had signed.⁸ Understanding this program illuminates both the scale of resources historically required and the ongoing legacy concerns.

Scale and organization:

• Biopreparat↗📖 reference★★★☆☆WikipediaBiopreparat: Soviet Biological Warfare AgencyRelevant historical reference for AI safety researchers studying dual-use technology risks, state-level misuse of emerging science, and governance failures in preventing catastrophic biological threats.Biopreparat was a covert Soviet agency (1974–1992) that ran the world's largest offensive biological weapons program, employing 30–40,000 personnel across ostensibly civilian re...biosecuritydual-use-researchexistential-riskgovernance+2Source ↗ was reportedly created in April 1974 as a civilian cover organization¹⁰
• Reportedly employed 30,000–40,000 personnel across some 40–50 research facilities, according to accounts by former program insiders¹⁰
• Reportedly included five major military-focused research institutes, numerous design facilities, three pilot plants, and five dual-use production plants¹⁰
• Annual production capacity for weaponized smallpox was reportedly on the order of 90–100 tons, according to defector accounts¹¹

Agents developed:

• Weaponized smallpox (reportedly continued even after WHO declared global eradication)
• Anthrax (including strains developed as enhanced "battle" variants)
• Plague, Q fever, tularemia, glanders, and Marburg hemorrhagic fever
• Agents reportedly designed for aerosol dispersal via ballistic or cruise missiles¹⁰

The Sverdlovsk incident (1979): An accidental release of anthrax spores from a Soviet military facility in Sverdlovsk reportedly killed at least 68 people; the true number remains uncertain because KGB records were reportedly destroyed.¹² The Soviet government initially attributed deaths to contaminated meat; Boris Yeltsin publicly acknowledged the military origin in 1992.¹²

Key defectors who revealed the program:

• Vladimir Pasechnik (defected 1989): Described as a high-level defector to the UK; his reported testimony enabled Western leaders to pressure Gorbachev about the program's scope¹³
• Ken Alibek (Kanatjan Alibekov, defected 1992): Described as a former first deputy director of Biopreparat; after emigrating he reportedly provided US government with a detailed accounting of the program, including work on tularemia and enhanced anthrax strains¹¹

Legacy concerns:

• Some facilities and scientists were reportedly absorbed into public health institutions after the USSR's dissolution
• US programs attempted to redirect former weapons scientists to peaceful research
• According to contemporaneous reporting, in late 1997 the US expanded efforts after detecting what officials described as intensified attempts by Iran and other states to acquire biological expertise from former Soviet institutes¹⁴

Lesson for AI risk: Even with massive state resources, Biopreparat reportedly required decades and thousands of scientists to develop reliable weapons. This suggests the wet-lab barrier is formidable—but also that determined state actors with existing infrastructure could integrate AI assistance more easily than non-state actors starting from scratch.

Non-State Actor Attempts

The historical record of non-state biological attacks reveals consistent technical failures despite significant motivation and resources:

1984 Oregon Salmonella Attack (Rajneeshees)

• Members of the Rajneeshee religious commune deliberately contaminated restaurant salad bars in The Dalles, Oregon with Salmonella typhimurium
• According to CDC records, the attack caused 751 cases of food poisoning and 45 hospitalizations; there were no deaths¹⁵
• The attack occurred in 1984 and remains the largest confirmed bioterrorist attack in U.S. history¹⁵
• It used a readily available pathogen requiring no sophisticated laboratory technology
• Key insight: Demonstrated that biological attacks don't require advanced technology, but also that impact was limited without sophisticated delivery

Aum Shinrikyo (1990s)

• Japanese cult with reportedly $1 billion in assets, hundreds of members, and PhD-level scientists¹⁶
• Attempted anthrax, botulinum toxin, and other biological agents—all efforts reportedly failed to produce casualties¹⁶
• An anthrax sprayer reportedly deployed in Tokyo produced no casualties, attributed partly to use of a vaccine strain by mistake¹⁶
• The group eventually succeeded with a sarin chemical attack in the Tokyo subway in 1995, killing 13 people and injuring thousands¹⁷
• Key insight: Even well-funded, technically sophisticated groups with scientific personnel have failed at biological weapons. The wet-lab barrier is real.

2001 Anthrax Letters (Amerithrax)

• Letters containing anthrax spores killed 5 people and infected 17 others in the United States¹⁸
• The FBI concluded the perpetrator was Bruce Ivins, a senior scientist at USAMRIID with decades of anthrax research experience and legitimate institutional access to spores¹⁸
• Key insight: An insider threat—not information access—enabled this attack. The perpetrator already possessed world-class expertise; AI would not have been the limiting factor.

Why has catastrophic bioterrorism not occurred?

This historical record could indicate either genuine difficulty (the barriers are high) or luck (we've been fortunate). The precautionary argument is that AI could systematically lower multiple barriers simultaneously, changing the calculus even if each individual barrier remains partially intact.

Current Biosecurity Infrastructure

DNA synthesis companies already screen orders for dangerous sequences, but screening isn't comprehensive:

DNA Synthesis Screening: The Critical Chokepoint

DNA synthesis screening is considered the key "chokepoint" in the AI-assisted bioweapons pipeline—if dangerous sequences can be intercepted before synthesis, attacks become much harder. However, significant gaps remain:

Current limitations:

• Participation in the International Gene Synthesis Consortium (IGSC) is voluntary—not all companies are members
• Regulations are inconsistent between countries
• Screening relies on matching against databases of known dangerous sequences—novel variants can evade detection
• High false positive rates require expensive human review
• Benchtop DNA synthesizers are emerging that could bypass commercial screening entirely

Post-Microsoft patch status: After research revealed high evasion rates against existing screening tools, a software patch was deployed to synthesis companies. According to reporting on that effort, the fix reportedly now catches approximately 97% of threats—but experts have cautioned that the fix remains incomplete and gaps persist.²⁰

Policy response: In April 2024, the White House OSTP released a Framework for Nucleic Acid Synthesis Screening↗🏛️ governmentFramework for Nucleic Acid Synthesis ScreeningRelevant to AI safety discussions around dual-use technology governance; this framework models how governments can establish screening and safeguards for powerful biotechnologies, offering lessons for analogous AI governance challenges.The Biden White House Office of Science and Technology Policy (OSTP) released a framework establishing standards for screening nucleic acid synthesis orders to prevent misuse fo...biosecuritygovernancepolicyexistential-risk+4Source ↗, requiring federally funded programs to screen customers and orders, keep records, and report suspicious orders.[^52] NIST is partnering with stakeholders to improve screening standards and mitigate AI-specific risks.[^52]

Emerging Defensive Infrastructure

SecureDNA: A Swiss foundation↗🔗 webSecureDNA – Biosecurity Screening for DNA SynthesisSecureDNA is a concrete technical intervention for bioweapon risk reduction, relevant to AI safety discussions around dual-use technology governance and proactive safety infrastructure for transformative technologies.SecureDNA is a Swiss nonprofit foundation developing cryptographic screening technology to prevent the synthesis of dangerous pathogens and bioweapons via DNA synthesis provider...biosecuritydual-use-researchexistential-riskgovernance+4Source ↗ providing free, privacy-preserving DNA synthesis screening that already exceeds 2026 regulatory requirements. SecureDNA screens sequences below the 50 base pair length using a "random adversarial threshold" algorithm designed to be more robust against AI-designed evasion.

Nucleic Acid Observatory (NAO): A collaboration between SecureBio and MIT↗🔗 webcollaboration between SecureBio and MITRelevant to AI safety audiences concerned with biological x-risks and dual-use threats; NAO's metagenomic surveillance infrastructure could also apply to detecting engineered or AI-assisted pathogen releases.The Nucleic Acid Observatory (NAO), a collaboration between SecureBio and MIT's Sculpting Evolution group, develops pathogen-agnostic biosurveillance systems capable of detectin...biosecurityexistential-risktechnical-safetyevaluation+2Source ↗ pioneering pathogen-agnostic early warning through deep metagenomic sequencing. Unlike traditional surveillance that looks for known pathogens, NAO aims to detect new and unknown pathogens through wastewater and pooled nasal swab sampling. SecureBioOrganizationSecureBioA biosecurity nonprofit applying the Delay/Detect/Defend framework to protect against catastrophic pandemics, including AI-enabled biological threats.Quality: 65/100's "Delay, Detect, Defend" strategy: Kevin Esvelt's SecureBio organization↗🔗 webSecureBio organizationSecureBio is a key organization in the biosecurity-AI intersection space, relevant to AI safety discussions about preventing LLMs and AI tools from enabling catastrophic biological harms.SecureBio is an organization focused on reducing biological risks, particularly those arising from advances in biotechnology and AI-enabled capabilities. They conduct research a...biosecurityexistential-riskgovernancepolicy+4Source ↗ works on multiple defensive layers:

• Delay: Synthesis screening and access controls
• Detect: Early warning systems like the NAO
• Defend: Societal resilience through germicidal UV light, pandemic-proof PPE stockpiles, and rapid countermeasure development

Emerging Technologies of Concern

Several emerging technologies could compound AI-enabled biosecurity risks by removing barriers that currently limit attack feasibility:

Benchtop DNA Synthesizers

A new generation of desktop DNA synthesis devices may enable users to print DNA in their own laboratories, potentially bypassing commercial screening entirely.

Current products:

• Kilobaser↗🔗 webKilobaser: Personal DNA & RNA SynthesizerRelevant to AI safety and biosecurity discussions as a real-world example of dual-use biotechnology: decentralized DNA synthesis could circumvent traditional biosecurity screening measures applied by commercial oligo suppliers.Kilobaser offers a compact, microfluidic chip-based desktop DNA and RNA synthesizer ('Kilobaser one-Xt') that enables on-demand custom oligonucleotide synthesis in under two hou...biosecuritydual-use-researchgovernanceexistential-risk+2Source ↗: Personal DNA/RNA synthesizer, reportedly measuring 27×33×33 cm, producing oligos in approximately 30–50 minutes with around 2.5 min/base turnaround, according to manufacturer specifications
• DNA Script SYNTAX System↗🔗 webDNA Script SYNTAX SystemThis product page is relevant to AI safety adjacent biosecurity discussions, particularly concerns about how decentralized DNA synthesis technology could circumvent screening protocols designed to prevent synthesis of dangerous pathogens or bioweapons-relevant sequences.The SYNTAX System by DNA Script is a benchtop enzymatic DNA synthesis (EDS) platform that enables rapid, on-demand synthesis of custom oligonucleotides without traditional phosp...biosecuritydual-use-researchexistential-riskgovernance+3Source ↗: Enzymatic DNA synthesis (water-based, avoiding harsh chemicals), reportedly supporting 96 parallel oligos up to 120 nucleotides per the company's published materials
• Evonetix Evaleo↗🔗 webEvonetix Evaleo Gene Synthesis PlatformRelevant to AI safety and biosecurity communities tracking dual-use biotechnology; advances in gene synthesis are a key risk factor in biological catastrophe scenarios, and this platform represents the state of the art in the field.Evonetix's Evaleo is a silicon chip-based DNA synthesis platform designed for high-fidelity, scalable gene synthesis. The technology aims to improve accuracy and throughput in s...biosecuritydual-use-researchexistential-riskgovernance+3Source ↗: Gene-length DNA synthesis on silicon chips, with the company claiming speeds approximately 10× faster than current technologies
• BioXp (Telesis Bio): Commercial benchtop synthetic biology workstation automating pipetting, mixing, thermal cycling, purification, and storage

Current limitations:

• According to some sources, most benchtop devices are limited to sequences under 120 base pairs—insufficient for most dangerous applications
• Not yet viable alternatives to centralized DNA providers for gene-length sequences
• Quality control and yield often inferior to commercial synthesis Biosecurity implications:
• NTI analysis↗🔗 web★★★★☆Nuclear Threat InitiativeBenchtop DNA Synthesis Devices: Capabilities, Biosecurity Implications, and GovernanceRelevant to AI safety discussions around dual-use technology governance and the challenge of maintaining safety norms as powerful tools become more accessible; offers a biotech parallel to AI proliferation governance debates.This NTI report examines how emerging benchtop DNA synthesis devices threaten to decentralize DNA production, potentially circumventing existing biosecurity screening protocols ...biosecuritygovernancedual-use-researchexistential-risk+3Source ↗ reportedly notes that "three converging technological trends—enzymatic synthesis, hardware automation, and increased demand from computational tools—are likely to drive rapid advancement in benchtop capabilities over the next decade"
• Manufacturers should implement rigorous sequence screening for each fragment produced
• Governments should provide clear regulations for manufacturers to incorporate screening
• Once capabilities exceed current limits, benchtop devices could become a significant biosecurity gap

Cloud Laboratories

Cloud laboratories↗🔗 web★★★★☆RAND CorporationDocumenting Cloud Labs and Examining How Remotely Operated Automated Laboratories Could Enable Bad ActorsRelevant to AI safety discussions around AI-enabled biological risk; provides empirical grounding on cloud lab infrastructure that could factor into debates over AI governance, biosecurity regulation, and dual-use research oversight.This RAND paper surveys 15 cloud laboratory organizations worldwide—remotely operated, automated research facilities representing AI-biotech convergence—and analyzes how these p...biosecuritydual-use-researchgovernancecapabilities+6Source ↗ are heavily automated, centralized research facilities where scientists run experiments remotely from computers. They present unique biosecurity challenges:

How cloud labs lower barriers:

• Reduce technical skill requirements by automating complex procedures
• Enable "one-stop-shop" research that could expand the pool of capable actors
• Allow experiments to be performed remotely, potentially bypassing ethical constraints in traditional academic settings
• Researchers retain full control over experimental design without physical presence

Current governance gaps:

• No public data on cloud lab operations, workflows, customer numbers, or locations worldwide
• No standardized approaches for customer screening shared between organizations
• Cybersecurity laws don't account for unique vulnerabilities of biological data and lab automation systems
• Biosafety regulations typically neglect digital threats like remote manipulation of synthesis machines

Proposed solutions (RAND↗🔗 web★★★★☆RAND CorporationRobust Biosecurity Measures Should Be Standardized at Scientific Cloud LabsRelevant to AI safety discussions around biological risks and governance of automated systems that lower barriers to dangerous research; published by RAND in November 2024 amid growing concern about AI-enabled bioweapon development.This RAND commentary examines the emerging risks posed by scientific cloud labs—remotely operated, highly automated laboratory facilities—and argues for standardized biosecurity...biosecuritygovernancedual-use-researchpolicy+4Source ↗):

• Create a Cloud Lab Security Consortium (CLSC) modeled on the International Gene Synthesis Consortium (IGSC) for DNA synthesis
• Minimum security standards: customer screening, controlled substance access, experiment screening, secured networks
• Human-in-the-loop controls when AI systems place synthesis orders for sequences of concern

Biological Design Tools (BDTs)

Beyond LLMs, specialized biological design tools present distinct risks:

AlphaFold↗🔗 web★★★★☆Google AIAlphaFold 3 predicts the structure and interactions of all of life’s moleculesRelevant to AI safety discussions around dual-use biological AI capabilities; AlphaFold 3's restricted access policy reflects ongoing tensions between open science and biosecurity risk management.Google DeepMind AlphaFold team, Isomorphic Labs (2024)Google DeepMind and Isomorphic Labs introduce AlphaFold 3, an AI model that extends beyond protein structure prediction to model the structure and interactions of DNA, RNA, liga...capabilitiesbiosecuritydual-use-researchexistential-risk+3Source ↗ and protein structure prediction:

• Revolutionary tool for predicting protein structure from genetic sequence; according to some sources, achieving over 90% accuracy on benchmark datasets
• Could enable optimization of existing hazards: increasing toxicity, improving immune evasion, enhancing transmissibility
• Could potentially enable design of completely novel toxins targeting human proteins
• Google DeepMind reportedly engaged more than 50 domain experts in biosecurity assessment during development of AlphaFold 3, according to published accounts
• Implements experimental refusal mechanisms to block misuse—but biological design often resides in dual-use space

Other BDT concerns:

• Machine learning for prediction of host range, transmissibility, and virulence
• Generative models for novel agent design
• Tools that help design sequences evading DNA screening (as demonstrated in published Microsoft research)

Dual-use nature: Unlike LLM guardrails, where harmful requests are often clearly distinguishable, biological design tool queries are frequently dual-use. The same protein optimization that could enhance a therapeutic could theoretically enhance a toxin. This makes technical controls more difficult than for text-based LLMs.

Policy recommendations (UNICRI↗📋 reportUNICRI: Dual-Use Research and Artificial IntelligencePublished by UNICRI (UN Interregional Crime and Justice Research Institute) in 2021, this report is relevant to AI governance discussions around how powerful AI tools in biology may require international oversight to prevent catastrophic misuse.This UNICRI report examines the dual-use risks of AI-driven protein-folding prediction tools like AlphaFold, which offer major benefits for medicine but could be weaponized to e...biosecuritydual-use-researchgovernanceexistential-risk+5Source ↗):

• Prerelease evaluation requirements for advanced biological models regardless of funding source
• Prioritize mitigating risks of pathogens capable of causing major epidemics
• Preserve researcher autonomy while implementing targeted controls on highest-risk capabilities

Research Governance & International Law

AI-enabled bioweapons risk exists within a broader context of biosecurity challenges, including ongoing debates about research oversight and international governance gaps.

Gain-of-Function and Enhanced Pandemic Pathogen Research

Gain-of-function (GoF) research—experiments that enhance pathogen transmissibility, virulence, or host range—has become intensely controversial, with implications for AI-biosecurity debates:

Recent policy developments:

• May 2024: The White House Office of Science and Technology Policy released the "Policy for Oversight of Dual Use Research of Concern and Pathogens with Enhanced Pandemic Potential" (DURC/PEPP Policy).[^52]
• May 2025: An executive order reportedly blocked the 2024 policy the day before it was scheduled to take effect.[^53]
• Ongoing: NIH reportedly identified more than 40 projects that may meet definitions of dangerous GoF research and, according to some sources, demanded scientists suspend work.[^54]

Congressional activity:

• The House approved a ban on federal funding for GoF research modifying risky pathogens.
• Scientific groups warn that vaguely worded provisions could unintentionally halt flu vaccine development and other beneficial research.
• The Risky Research Review Act (S. 854, H.R. 1864) would establish a life sciences research security board.

Key limitation: Both the 2014 DURC Policy and the 2024 PEPP Policy apply only to government-funded research. Extending coverage to privately funded research would require new regulations or legislation. AI labs developing biological design tools with private funding currently face no equivalent oversight requirements.

Relevance to AI risk: The GoF debate previews challenges AI governance will face:

• Distinguishing beneficial from dangerous research is difficult.
• Oversight mechanisms are primarily voluntary and apply only to government-funded work.
• International coordination is lacking.
• Technical definitions ("gain of function," "enhanced pandemic potential") are contested.

The Biological Weapons Convention: Structural Weaknesses

The Biological Weapons Convention (BWC), opened for signature in 1972, prohibits the development, production, and stockpiling of biological weapons.[^55] As of the most recent review cycle it has 187 states parties.[^56] Despite its broad membership, the treaty has significant structural weaknesses.

No verification regime:

• Unlike chemical and nuclear weapons agreements, the BWC contains no formal verification provisions.
• Attempts to develop a verification protocol collapsed in 2001 after years of negotiation.[^57]
• According to some analysts, governments effectively ceased substantive discussion of verification within the treaty framework for over two decades following that failure.

Minimal institutional support:

• The BWC Implementation Support Unit has only four staff members.[^58]
• Its budget is, according to some sources, smaller than that of an average McDonald's restaurant—a comparison attributed to philosopher Toby OrdPersonToby OrdComprehensive biographical profile of Toby Ord documenting his 10% AI extinction estimate and role founding effective altruism, with detailed tables on risk assessments, academic background, and in...Quality: 41/100.[^59]
• By contrast, the IAEA employs more than 2,500 staff and the OPCW more than 500 staff.[^60]

Recent developments:

• December 2022: States Parties established a Working Group on Strengthening the Convention.[^61]
• 2024: The fourth and fifth Working Group sessions were held in August and December 2024.
• December 2024: The fifth session reportedly "ended with a regrettable conclusion in which a single States Party undermined the noteworthy progress achieved"—a setback described by the Council on Strategic Risks.[^62]
• The Working Group has reportedly only seven days of scheduled time through the end of 2025 allocated specifically for verification discussion.[^63]

Practical limitations:

• No politically palatable, technologically feasible, and financially sustainable system can guarantee detection of all biological weapons programs.
• Rapid advances in biotechnology create new verification challenges.
• AI capabilities could make verification even more difficult by enabling novel agent design.

What's possible: While perfect verification is unachievable, analysts including those writing in the Bulletin of the Atomic Scientists have argued that "measures in combination could generate considerably greater confidence in compliance by BWC states parties."[^64]

Defensive Technologies and Pandemic Preparedness

The same technological advances that could enable attacks also offer powerful defensive capabilities. Many experts believe defense will ultimately win the offense-defense balance—the question is whether we're in a dangerous transition period.

mRNA Vaccine Platforms

The COVID-19 pandemic demonstrated the transformative potential of mRNA vaccines for rapid response:

Speed advantages:

• Traditional vaccines require time-consuming manufacturing with live pathogens
• mRNA vaccines can be designed in days once a pathogen's genetic sequence is known[^65]
• COVID-19 mRNA vaccines received FDA Emergency Use Authorization in under one year—unprecedented in vaccine history[^66]
• CEPI's "100 Days Mission" aims to develop safe, effective vaccines against novel threats within 100 days of a pandemic being declared[^67]

Manufacturing advantages:

• Cell-free manufacture enables accelerated, scalable production
• Standardizable processes require minimal facility adaptations between products
• Smaller manufacturing footprints than traditional vaccines
• Same facility can produce multiple vaccine products

Safety profile:

• mRNA does not enter the cell nucleus and cannot integrate into the cellular genome[^68]
• Can be administered repeatedly without triggering anti-vector immunity (unlike viral vector vaccines)
• Avoids live pathogen handling in manufacturing

Pandemic preparedness implications:

• Platform is "pathogen-agnostic"—the same technology works against any target with a known sequence
• BARDA and CEPI are reportedly supporting development of dozens of vaccine candidates against high-risk pathogens
• Next-generation "trans-amplifying" mRNA vaccines↗🔗 web★★★★☆CEPINext-generation "trans-amplifying" mRNA vaccinesRelevant to AI safety researchers interested in biosecurity and dual-use technology governance; ta-mRNA platforms exemplify how powerful emerging biotechnologies require careful oversight due to their potential for both beneficial pandemic response and misuse.CEPI (Coalition for Epidemic Preparedness Innovations) announces research into trans-amplifying mRNA (ta-mRNA) vaccines, a next-generation platform that could produce stronger i...biosecuritygovernanceexistential-riskpolicy+2Source ↗ under development could provide stronger immune responses at lower doses

For AI-bioweapons specifically: Rapid vaccine development could limit the damage from engineered pathogens if detected early. However, novel agents designed to evade detection or existing countermeasures would still pose severe risks during the response window.

Metagenomic Surveillance

Traditional disease surveillance looks for known pathogens. Metagenomic sequencing offers pathogen-agnostic detection:

How it works:

• Deep sequencing of all genetic material in samples (wastewater, nasal swabs, etc.)
• Computational analysis identifies viral, bacterial, and other sequences
• Can detect novel or unexpected pathogens that would not be caught by targeted testing

Current research:

• Nucleic Acid Observatory (NAO)↗🔗 webcollaboration between SecureBio and MITRelevant to AI safety audiences concerned with biological x-risks and dual-use threats; NAO's metagenomic surveillance infrastructure could also apply to detecting engineered or AI-assisted pathogen releases.The Nucleic Acid Observatory (NAO), a collaboration between SecureBio and MIT's Sculpting Evolution group, develops pathogen-agnostic biosurveillance systems capable of detectin...biosecurityexistential-risktechnical-safetyevaluation+2Source ↗: Sequencing wastewater from major US airports and treatment plants to establish pathogen-agnostic baselines
• One published dataset comprised reportedly 13.1 terabases sequenced from 20 wastewater samples collected at the Los Angeles Hyperion treatment plant, which serves approximately 4 million residents[^69]
• A Lancet Microbe publication↗🔗 webLancet Microbe publicationPublished by the Nucleic Acid Observatory, a project focused on biosurveillance as a defense against pandemic-level biological risks; relevant to AI safety adjacent discussions of catastrophic and existential biological risks.This blog post from the Nucleic Acid Observatory (NAO) announces a publication in Lancet Microbe presenting their work on using metagenomic sequencing of environmental samples f...biosecurityexistential-riskgovernancepolicy+2Source ↗ established sensitivity models for wastewater metagenomic sequencing (W-MGS) detection

Sensitivity and cost tradeoffs:

• Untargeted shotgun sequencing is less sensitive than targeted methods for known pathogens
• Hybridization capture panels can greatly increase sensitivity for viruses included in the panel, but may reduce sensitivity to entirely unknown pathogens
• Large variation in viral detection exists based on sewershed hydrology and laboratory protocols
• Modeled sensitivity for certain bacterial pathogens has been estimated at roughly 1 infected person detectable among 257–2,250 individuals in a sewershed, according to published sensitivity analyses[^70]

For AI-bioweapons specifically: Metagenomic surveillance could provide early warning for engineered pathogens that evade targeted detection. However, sensitivity limits mean outbreaks may need to reach significant scale before detection occurs.

Far-UVC Germicidal Light

Far-UVC↗📖 reference★★★☆☆WikipediaFar-UVC Light TechnologyRelevant to biosecurity and pandemic preparedness discussions in AI safety contexts; far-UVC represents a potential physical countermeasure against engineered pathogens, and understanding its capabilities and limitations is useful for assessing biological risk mitigation strategies.Far-UVC (ultraviolet-C light at 207-222 nm wavelengths) is a disinfection technology that can inactivate pathogens including viruses and bacteria in occupied spaces without the ...biosecurityexistential-riskgovernancepolicy+1Source ↗ light, operating in the 200–235 nm wavelength range, is emerging as a potentially transformative technology for airborne pathogen inactivation in occupied spaces[^71]:

Why it's different from conventional UV:

• Conventional germicidal UV-C (254 nm) harms human skin and eyes—restricting its use to upper-room applications or unoccupied spaces
• Far-UVC (typically 222 nm) is absorbed in the outer dead layer of human skin and in the tear layer of the eyes, and cannot penetrate to living tissue[^72]
• This property enables direct disinfection of the breathing zone while people are present

Efficacy:

• A very low dose of 2 mJ/cm² of 222-nm light has been reported to inactivate more than 95% of airborne H1N1 influenza virus in laboratory conditions[^73]
• Studies suggest a single far-UVC fixture can deliver the equivalent of 33–66 air changes per hour for pathogen removal[^74]
• Far-UVC has been tested against tuberculosis, SARS-CoV-2, influenza, and murine norovirus, with reported reductions of up to 99.8% for murine norovirus[^75]
• A 2025 review characterized far-UVC as having "high ability" to kill pathogens with a "high level of safety," though the authors noted that long-term human exposure data remain limited[^76]

Applications for pandemic preparedness:

• Installation in hospitals, schools, airports, and public transit could dramatically reduce airborne transmission
• Blueprint Biosecurity↗🔗 web★★★★☆Blueprint BiosecurityBlueprint BiosecurityRelevant to biosecurity and pandemic preparedness communities; far-UVC is a proposed non-pharmaceutical intervention for reducing airborne pathogen transmission, with potential relevance to future pandemic response strategies including those involving engineered pathogens.Blueprint Biosecurity announces $1M in EXHALE program grants to two research teams evaluating far-UVC light's effectiveness against real human-generated respiratory aerosols con...biosecuritypandemic-preparednessgovernancepolicy+2Source ↗ is reportedly funding research teams to evaluate deployment in real-world scenarios
• Coefficient GivingOrganizationCoefficient GivingCoefficient Giving (formerly Open Philanthropy) has directed $4B+ in grants since 2014, including $336M to AI safety (~60% of external funding). The organization spent ~$50M on AI safety in 2024, w...Quality: 55/100 has issued an RFI on far-UVC evaluation↗🔗 web★★★★☆Coefficient GivingRFI on far-UVC evaluationThis RFI is tangential to AI safety but relevant to Open Philanthropy's broader biosecurity work; it illustrates how the funder scopes and solicits input before making grants in emerging scientific domains like pandemic prevention technology.Open Philanthropy's 2023 RFI soliciting expert input on far-UVC light technology (200–240 nm) as a promising disinfection approach for reducing airborne pathogen transmission in...biosecurityevaluationpolicygovernance+2Source ↗
• NIST is reportedly collaborating with industry on standards development

Remaining questions:

• Long-term human exposure effects require further research
• Real-world efficacy in varied building environments is not yet fully characterized
• Cost and feasibility of widespread deployment remain open questions

For AI-bioweapons specifically: Far-UVC could provide a layer of defense against aerosol-dispersed biological agents in public spaces. Even if attackers successfully synthesize and deploy pathogens, widespread far-UVC installation could limit transmission and buy time for medical countermeasure deployment.

Mitigations

Model-Level Interventions

Refusals and filtering — Training models not to help with bioweapon development and filtering dangerous outputs. But these are imperfect—models can be jailbroken, fine-tuned, or open-source models may lack restrictions entirely.

Effectiveness assessment:

• Reduces casual misuse
• Raises barrier for unsophisticated actors
• Does not prevent determined actors with technical skills
• Cannot address open-source model proliferation

Evaluations before deployment — Testing models for biosecurity risks during development, as part of responsible scaling policies. Useful but relies on labs' good faith and competence.

AI-Specific Governance

Compute governance — Limiting who can train powerful models reduces the availability of capable models to bad actors. Information security around model weights becomes important if models can provide meaningful uplift.

Biological capability thresholds — Anthropic's RSP and similar frameworks establish biological capability as a key threshold for enhanced safety measures. This creates systematic evaluation requirements.

Open-source restrictions — Limiting the release of model weights for systems with significant biological knowledge. Controversial due to benefits of open research.

Broader Biosecurity Measures

Broader biosecurity measures may matter more than AI-specific interventions:

DNA synthesis screening — Flagging dangerous sequences before synthesis. The primary defense but has significant gaps that AI can exploit.

Laboratory access controls — Restricting who can work with dangerous pathogens. Effective for legitimate facilities; doesn't address improvised labs.

Disease surveillance — Early detection of outbreaks. Benefits from AI advances and may be where AI provides greatest defensive value.

Medical countermeasures — Rapid vaccine and treatment development. mRNA platforms demonstrated during COVID-19 show how quickly responses can be developed.

Timeline

Expert Perspectives

Expert opinion on AI-bioweapons risk is divided, with prominent voices on both sides:

Those More Concerned

Kevin Esvelt (MIT): One of the most vocal experts on AI-biosecurity risks. Esvelt emphasizes that if you ask a chatbot how to cause a pandemic, "it will suggest the 1918 influenza virus. It will even tell you where to find the gene sequences online and where to purchase the genetic components." He co-founded SecureDNA and SecureBio to address these risks.

Dario AmodeiPersonDario AmodeiComprehensive biographical profile of Anthropic CEO Dario Amodei documenting his competitive safety development philosophy, 10-25% catastrophic risk estimate, 2026-2030 AGI timeline, and Constituti...Quality: 41/100 (Anthropic CEO): In July 2023, stated there was a "substantial risk" that within 2-3 years, AI would "greatly widen the range of actors with the technical capability to conduct a large-scale biological attack." In February 2025, reported that DeepSeek was "the worst" model tested for biosecurity, generating information "that can't be found on Google or easily found in textbooks."

Johannes Heidecke (OpenAI Head of Safety Systems): In June 2025, announced OpenAI expects upcoming models to hit "high-risk classification" for biological capabilities. Emphasized that "99% or even one in 100,000 performance is [not] sufficient" for testing accuracy.

Rocco Casagrande (Gryphon Scientific): After red-teaming Claude, said he was "personally surprised and dismayed by how capable current LLMs were" and that "these things are developing extremely, extremely fast."

Toby Ord (Oxford): Estimates engineered pandemic risk at 1 in 30 by 2100—second highest anthropogenic existential risk after AI itself.

Georgia Adamson and Gregory C. Allen (CSIS): Their August 2025 report warns current U.S. biosecurity measures are "ill-equipped" to meet AI-enabled challenges, with BDT safeguards "already circumventable post-deployment."

Bill Drexel and Caleb Withers (CNAS): Their August 2024 report warns AI could enable "catastrophic threats far exceeding the impact of COVID-19."

Those More Skeptical

RAND researchers (Mouton, Lucas, Guest): Their 2024 study found "no statistically significant difference" between AI-assisted and non-AI groups in bioweapon planning capability. This is the strongest empirical evidence against immediate AI uplift concerns.

Some biosecurity practitioners: Emphasize that the wet lab bottleneck—tacit knowledge, equipment access, technique—remains the primary barrier, and AI cannot transfer hands-on skills.

Information abundance argument: Dangerous information is already accessible through scientific literature and the internet. AI may provide convenience but not fundamentally new capabilities.

The Disagreement Structure

The debate often reduces to different assessments of:

2025 shift: The debate has evolved significantly. Both major frontier labs now officially acknowledge their next-generation models pose elevated biological risks. The question is shifting from "does AI provide uplift?" to "how much uplift, and can mitigations keep pace?"

Notably: Even those who downplay current uplift often acknowledge that future models may pose greater risks, and that defensive investments are worthwhile regardless.

Sources & Resources

2025-2026 Key Sources

Primary Research

• RAND Corporation (2024): The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study↗🔗 web★★★★☆RAND CorporationRAND Corporation studyRAND research reports on AI and bioweapons risk are directly relevant to frontier AI evaluation policy, particularly debates around capability thresholds used in safety frameworks like Anthropic's RSP or OpenAI's preparedness framework.This RAND Corporation research report examines the risk of AI systems providing meaningful uplift to actors seeking to develop biological weapons, focusing on how to assess capa...existential-riskevaluationred-teamingcapabilities+6Source ↗ - The most rigorous empirical study of AI uplift to date
• Microsoft Research (2025): AI-designed toxins evade DNA screening - Published in Science, October 2025
• National Academies (2025): The Age of AI in the Life Sciences: Benefits and Biosecurity Considerations - Comprehensive government-commissioned study on AI biosecurity risks
• Gryphon Scientific (2023): Red-team evaluation of Claude's biological capabilities - Coverage in Semafor↗🔗 webAI-Assisted Bioterrorism Is Top Concern for OpenAI and AnthropicRelevant to discussions of catastrophic risk evaluation and responsible deployment policies at frontier AI labs; illustrates how biosecurity has become a key focus of AI safety red-teaming efforts.A Semafor news article reporting on concerns from OpenAI and Anthropic that AI systems could assist malicious actors in developing bioweapons, drawing on findings from Gryphon S...biosecurityexistential-riskred-teamingdeployment+4Source ↗
• UNICRI (2021): The Potential for Dual-Use of Protein-Folding Prediction↗📋 reportUNICRI: Dual-Use Research and Artificial IntelligencePublished by UNICRI (UN Interregional Crime and Justice Research Institute) in 2021, this report is relevant to AI governance discussions around how powerful AI tools in biology may require international oversight to prevent catastrophic misuse.This UNICRI report examines the dual-use risks of AI-driven protein-folding prediction tools like AlphaFold, which offer major benefits for medicine but could be weaponized to e...biosecuritydual-use-researchgovernanceexistential-risk+5Source ↗ - Early analysis of AlphaFold biosecurity implications
• Council on Strategic Risks (2023): The Cyber-Biosecurity Nexus↗🔗 webThe Cyber-Biosecurity NexusRelevant to AI safety audiences concerned with dual-use emerging technologies; AI-accelerated synthetic biology and automated lab systems are key threat multipliers discussed in this biosecurity-cybersecurity overlap analysis.This Council on Strategic Risks briefer examines the intersection of cybersecurity and biosecurity, identifying how advances in automation, synthetic biology democratization, an...biosecuritycybersecuritygovernancepolicy+4Source ↗

Policy and Analysis

• CSIS (2025): Opportunities to Strengthen U.S. Biosecurity from AI-Enabled Bioterrorism by Georgia Adamson and Gregory C. Allen
• CNAS (2024): AI and the Evolution of Biological National Security Risks↗🔗 web★★★★☆CNASAI and the Evolution of Biological National Security RisksA CNAS policy report providing a broad overview of AI-biosecurity intersection for policymakers; useful for understanding governance challenges around dual-use AI capabilities in the biological domain.This CNAS report examines how AI advancements intersect with biosecurity risks, analyzing threats from state actors, nonstate actors, and accidental releases. It assesses whethe...biosecuritydual-use-researchexistential-riskgovernance+5Source ↗ by Bill Drexel and Caleb Withers
• White House OSTP (2024): Framework for Nucleic Acid Synthesis Screening↗🏛️ governmentFramework for Nucleic Acid Synthesis ScreeningRelevant to AI safety discussions around dual-use technology governance; this framework models how governments can establish screening and safeguards for powerful biotechnologies, offering lessons for analogous AI governance challenges.The Biden White House Office of Science and Technology Policy (OSTP) released a framework establishing standards for screening nucleic acid synthesis orders to prevent misuse fo...biosecuritygovernancepolicyexistential-risk+4Source ↗
• White House OSTP (2024): Policy for Oversight of DURC and PEPP↗🏛️ governmentPolicy for Oversight of Dual Use Research of Concern and Pathogens with Enhanced Pandemic PotentialDirectly relevant to AI safety discussions around dual-use research governance; offers a regulatory model for managing high-risk research that may inform analogous AI oversight frameworks, particularly around capability thresholds and institutional review processes.This May 2024 U.S. federal policy establishes a unified oversight framework for dual use research of concern (DURC) and pathogens with enhanced pandemic potential (PEPPs), super...governancebiosecuritypolicydual-use-research+4Source ↗
• NIST/AISI (2024): Pre-deployment evaluation of Claude 3.5 Sonnet↗🏛️ government★★★★★NISTPre-deployment evaluation of Claude 3.5 SonnetThis is one of the first publicly disclosed government-conducted pre-deployment AI safety evaluations, setting a precedent for how regulatory bodies may assess frontier models before release; relevant to governance, capability evaluation, and red-teaming methodology discussions.The U.S. and UK AI Safety Institutes jointly conducted pre-deployment safety evaluations of Anthropic's upgraded Claude 3.5 Sonnet, testing biological capabilities, cyber capabi...evaluationred-teaminggovernancepolicy+6Source ↗
• Congressional Research Service: Oversight of Gain-of-Function Research with Pathogens: Issues for Congress↗🏛️ government★★★★★US CongressOversight of Gain-of-Function Research with Pathogens: Issues for CongressRelevant to AI safety discussions around dual-use research governance and oversight frameworks; offers a concrete legislative case study for how governments manage high-risk, high-reward research with catastrophic misuse potential.This CRS report provides a comprehensive analysis of federal oversight mechanisms for gain-of-function (GOF) research, which enhances pathogen transmissibility or virulence. It ...governancebiosecuritypolicyexistential-risk+4Source ↗

Industry Frameworks

• Anthropic: Responsible Scaling Policy↗🔗 web★★★★☆AnthropicResponsible Scaling PolicyThis is Anthropic's foundational policy document establishing how it gates deployment of increasingly capable models; a key reference for understanding industry-led AI governance frameworks and voluntary safety commitments.Anthropic introduces its Responsible Scaling Policy (RSP), a framework of technical and organizational protocols for managing catastrophic risks as AI systems become more capabl...governancepolicyai-safetycapabilities+6Source ↗
• Anthropic (2025): Biorisk Evaluations - Detailed methodology for Claude Opus 4 safety testing
• OpenAI: Preparedness Framework↗🔗 web★★★★☆OpenAIPreparedness FrameworkOpenAI's official institutional framework for catastrophic risk evaluation; relevant for understanding how leading AI labs operationalize safety policies and set deployment guardrails for frontier models.OpenAI's Preparedness Framework outlines a structured approach to evaluating and managing catastrophic risks from frontier AI models, including threats related to CBRN weapons, ...governancepolicyevaluationdeployment+6Source ↗
• OpenAI (2025): Preparing for Future AI Capabilities in Biology - High-risk classification announcement
• OpenAI (2024): Building an early warning system for LLM-aided biological threat creation↗🔗 web★★★★☆OpenAIBuilding an early warning system for LLM-aided biological threat creationAn OpenAI empirical study and methodological blueprint for assessing LLM-enabled bioweapon risk, directly tied to their Preparedness Framework; represents one of the first systematic attempts to quantify AI uplift in biological threat scenarios.OpenAI presents a methodology for evaluating whether LLMs like GPT-4 could meaningfully assist malicious actors in creating biological threats. In a controlled study with 100 pa...biosecurityevaluationred-teamingdual-use+6Source ↗
• Google DeepMind: Our approach to biosecurity for AlphaFold 3↗🔗 webOur approach to biosecurity for AlphaFold 3This is DeepMind's official biosecurity policy document accompanying the AlphaFold 3 release, relevant to discussions of dual-use AI governance and responsible deployment practices for high-capability biological AI systems.DeepMind outlines the biosecurity measures and risk mitigation strategies implemented for AlphaFold 3, addressing concerns about dual-use potential of a powerful protein structu...biosecuritygovernancedeploymentdual-use+5Source ↗

Biosecurity Organizations

• SecureDNA: DNA synthesis screening platform↗🔗 webSecureDNA – Biosecurity Screening for DNA SynthesisSecureDNA is a concrete technical intervention for bioweapon risk reduction, relevant to AI safety discussions around dual-use technology governance and proactive safety infrastructure for transformative technologies.SecureDNA is a Swiss nonprofit foundation developing cryptographic screening technology to prevent the synthesis of dangerous pathogens and bioweapons via DNA synthesis provider...biosecuritydual-use-researchexistential-riskgovernance+4Source ↗
• SecureBio: Pandemic preparedness organization↗🔗 webSecureBio organizationSecureBio is a key organization in the biosecurity-AI intersection space, relevant to AI safety discussions about preventing LLMs and AI tools from enabling catastrophic biological harms.SecureBio is an organization focused on reducing biological risks, particularly those arising from advances in biotechnology and AI-enabled capabilities. They conduct research a...biosecurityexistential-riskgovernancepolicy+4Source ↗
• Nucleic Acid Observatory: Pathogen-agnostic surveillance↗🔗 webcollaboration between SecureBio and MITRelevant to AI safety audiences concerned with biological x-risks and dual-use threats; NAO's metagenomic surveillance infrastructure could also apply to detecting engineered or AI-assisted pathogen releases.The Nucleic Acid Observatory (NAO), a collaboration between SecureBio and MIT's Sculpting Evolution group, develops pathogen-agnostic biosurveillance systems capable of detectin...biosecurityexistential-risktechnical-safetyevaluation+2Source ↗
• Nuclear Threat Initiative (NTI): Biosecurity resources↗🔗 web★★★★☆Nuclear Threat InitiativeBiosecurity resourcesNTI is a leading policy organization on nuclear and biological threats; this hub is relevant to AI safety due to its focus on AI-bio convergence risks and governance gaps in AI models used in life sciences.The Nuclear Threat Initiative (NTI) biosecurity program addresses biological threats from natural, accidental, and intentional sources, including the intersection of AI and biot...biosecuritygovernanceexistential-riskpolicy+5Source ↗
• Blueprint BiosecurityOrganizationBlueprint BiosecurityAn EA-funded biosecurity nonprofit founded in 2023 by Jake Swett, dedicated to achieving breakthroughs in pandemic prevention through far-UVC germicidal light, next-generation PPE.Quality: 60/100: Far-UVC research↗🔗 web★★★★☆Blueprint BiosecurityBlueprint Biosecurity - Far-UVC Research InitiativeRelevant to AI safety researchers interested in biosecurity as a parallel existential risk domain and in how defensive technologies can reduce catastrophic risk; Far-UVC is considered a promising scalable biosecurity intervention by several EA-adjacent organizations.Blueprint Biosecurity is an organization focused on biosecurity research and policy, with a particular emphasis on Far-UVC light technology as a scalable intervention to reduce ...biosecurityexistential-riskpolicygovernance+2Source ↗

Emerging Technologies

• NTI (2024): Benchtop DNA Synthesis Devices: Capabilities, Biosecurity Implications, and Governance↗🔗 web★★★★☆Nuclear Threat InitiativeBenchtop DNA Synthesis Devices: Capabilities, Biosecurity Implications, and GovernanceRelevant to AI safety discussions around dual-use technology governance and the challenge of maintaining safety norms as powerful tools become more accessible; offers a biotech parallel to AI proliferation governance debates.This NTI report examines how emerging benchtop DNA synthesis devices threaten to decentralize DNA production, potentially circumventing existing biosecurity screening protocols ...biosecuritygovernancedual-use-researchexistential-risk+3Source ↗
• RAND (2024): Documenting Cloud Labs and Examining How Remotely Operated Automated Laboratories Could Enable Bad Actors↗🔗 web★★★★☆RAND CorporationDocumenting Cloud Labs and Examining How Remotely Operated Automated Laboratories Could Enable Bad ActorsRelevant to AI safety discussions around AI-enabled biological risk; provides empirical grounding on cloud lab infrastructure that could factor into debates over AI governance, biosecurity regulation, and dual-use research oversight.This RAND paper surveys 15 cloud laboratory organizations worldwide—remotely operated, automated research facilities representing AI-biotech convergence—and analyzes how these p...biosecuritydual-use-researchgovernancecapabilities+6Source ↗
• RAND (2024): Robust Biosecurity Measures Should Be Standardized at Scientific Cloud Labs↗🔗 web★★★★☆RAND CorporationRobust Biosecurity Measures Should Be Standardized at Scientific Cloud LabsRelevant to AI safety discussions around biological risks and governance of automated systems that lower barriers to dangerous research; published by RAND in November 2024 amid growing concern about AI-enabled bioweapon development.This RAND commentary examines the emerging risks posed by scientific cloud labs—remotely operated, highly automated laboratory facilities—and argues for standardized biosecurity...biosecuritygovernancedual-use-researchpolicy+4Source ↗
• EMBO Reports (2024): Security challenges by AI-assisted protein design↗🔗 webSecurity challenges by AI-assisted protein designPublished in EMBO Reports, this paper is relevant to AI safety discussions around catastrophic biological risks and how AI capabilities in scientific domains require proactive governance before widespread misuse becomes feasible.This paper examines the dual-use risks emerging from AI-powered protein design tools, analyzing how advances in computational biology could be exploited to engineer harmful biol...biosecuritydual-use-researchexistential-riskgovernance+5Source ↗

International Governance

• Arms Control Association: The Biological Weapons Convention (BWC) At A Glance↗🔗 webBiological Weapons ConventionRelevant for AI safety researchers interested in biosecurity governance as a precedent and model for regulating dual-use technologies with catastrophic potential, including comparisons to AI governance frameworks.An overview of the Biological Weapons Convention (BWC), the international treaty prohibiting the development, production, and stockpiling of biological weapons. The resource cov...biosecuritygovernanceexistential-riskpolicy+3Source ↗
• Arms Control Association (2024): Strengthening the Biological Weapons Convention↗🔗 webStrengthening the Biological Weapons ConventionRelevant to AI safety governance discussions as a case study in the challenges of multilateral arms control verification and compliance regimes for dual-use technologies, offering lessons for emerging AI governance frameworks.This analysis examines the BWC working group established in 2022 to strengthen the treaty across seven areas including verification, compliance, and scientific developments. Mid...governancebiosecuritypolicyexistential-risk+2Source ↗
• Bulletin of the Atomic Scientists (2024): How the Biological Weapons Convention could verify treaty compliance↗🔗 webBulletin of the Atomic Scientists arguesRelevant to AI safety as biosecurity governance intersects with AI-enabled bioweapon risks; illustrates challenges of verifying compliance in dual-use technology domains, a problem analogous to AI treaty verification.This Bulletin of the Atomic Scientists analysis examines the longstanding absence of verification mechanisms in the 1972 Biological Weapons Convention (BWC), explores why past e...biosecuritygovernancepolicyexistential-risk+3Source ↗
• Council on Strategic Risks (2025): Derailment of the Fifth Working Group of the BWC↗🔗 webDerailment of the Fifth Working Group of the Biological and Toxin Weapons Convention - The Council on Strategic RisksRelevant to AI safety researchers interested in international governance failures; the BTWC breakdown illustrates challenges in coordinating global oversight of dual-use technologies, a dynamic directly analogous to emerging AI governance challenges.This article from the Council on Strategic Risks examines the failure or obstruction of the Fifth Working Group of the Biological and Toxin Weapons Convention (BTWC), a key mult...biosecuritygovernancepolicyexistential-risk+3Source ↗

Defensive Technologies

• Nature (2018): Far-UVC light: A new tool to control the spread of airborne-mediated microbial diseases↗📄 paper★★★★★Nature (peer-reviewed)Far-UVC light: A new tool to control the spread of airborne-mediated microbial diseasesScientific study on far-UVC light technology for pathogen inactivation; relevant to AI safety as biosecurity research addressing pandemic risks that could inform AI governance priorities and resource allocation in dual-use technology oversight.David Welch, Manuela Buonanno, Veljko Grilj et al. (2017)1 citationsThis study demonstrates that far-UVC light (207-222 nm) can efficiently inactivate airborne viruses and bacteria without harming human skin or eyes, unlike conventional UVC ligh...biosecuritydual-use-researchx-riskSource ↗
• Scientific Reports (2024): 222 nm far-UVC light markedly reduces infectious airborne virus in an occupied room↗📄 paper★★★★★Nature (peer-reviewed)222 nm far-UVC light markedly reduces infectious airborne virus in an occupied roomThis empirical study on far-UVC light's efficacy against airborne viruses is relevant to AI safety biosecurity research, demonstrating technological interventions for pandemic preparedness and occupational safety in high-risk settings.Norman Kleiman, David Welch, Raabia Hashmi et al. (2023)This study demonstrates that 222 nm far-UVC light effectively reduces infectious airborne viruses in occupied indoor spaces. Researchers installed four 222-nm light fixtures in ...biosecuritydual-use-researchx-riskSource ↗
• Lancet Microbe (2025): Inferring the sensitivity of wastewater metagenomic sequencing for virus detection↗🔗 webLancet Microbe publicationPublished by the Nucleic Acid Observatory, a project focused on biosurveillance as a defense against pandemic-level biological risks; relevant to AI safety adjacent discussions of catastrophic and existential biological risks.This blog post from the Nucleic Acid Observatory (NAO) announces a publication in Lancet Microbe presenting their work on using metagenomic sequencing of environmental samples f...biosecurityexistential-riskgovernancepolicy+2Source ↗
• Virology Journal (2025): Revolutionizing immunization: a comprehensive review of mRNA vaccine technology↗🔗 web★★★★☆Springer (peer-reviewed)Revolutionizing immunization: a comprehensive review of mRNA vaccine technologyA peer-reviewed journal article on mRNA vaccine technology with potential relevance to AI safety discussions around biotechnology risks, gain-of-function research governance, and dual-use concerns in synthetic biology.Kai Yuan Leong, Seng Kong Tham, Chit Laa Poh (2025)60 citations · Virology Journalbiosecuritydual-use-researchx-riskSource ↗

Historical Background

• Wikipedia: Soviet biological weapons program↗📖 reference★★★☆☆WikipediaSoviet biological weapons programRelevant background for AI safety researchers studying biosecurity risks, state-level misuse of dual-use technologies, and the failures of international arms control treaties to prevent covert WMD programs.Documents the Soviet Union's covert and massive biological weapons program spanning from the 1920s to at least 1992, operated under the civilian cover organization Biopreparat i...biosecurityexistential-riskgovernancedual-use-research+3Source ↗
• Wikipedia: Biopreparat↗📖 reference★★★☆☆WikipediaBiopreparat: Soviet Biological Warfare AgencyRelevant historical reference for AI safety researchers studying dual-use technology risks, state-level misuse of emerging science, and governance failures in preventing catastrophic biological threats.Biopreparat was a covert Soviet agency (1974–1992) that ran the world's largest offensive biological weapons program, employing 30–40,000 personnel across ostensibly civilian re...biosecuritydual-use-researchexistential-riskgovernance+2Source ↗
• PMC (2023): The History of Anthrax Weaponization in the Soviet Union↗🏛️ government★★★★☆PubMed Central (peer-reviewed)The History of Anthrax Weaponization in the Soviet UnionHistorical analysis of Soviet bioweapon development programs that examines dual-use research and the relationship between offensive bioweapon development and civilian applications, relevant to understanding biosecurity risks and historical precedents for AI safety governance.Ioannis Nikolakakis, Spyros N Michaleas, George Panayiotakopoulos et al. (2023)6 citations · CureusThis historical paper examines the Soviet Union's anthrax weaponization program and its broader implications for biowarfare research and public health. The authors document how ...biosecuritydual-use-researchx-riskSource ↗
• Toby Ord: The Precipice: Existential Risk and the Future of Humanity (2020)

General Context

• 80,000 HoursOrganization80,000 Hours80,000 Hours is the largest EA career organization, reaching 10M+ readers and reporting 3,000+ significant career plan changes, with 80% of $10M+ funding from Coefficient Giving. Since 2016 they've...Quality: 45/100: Problem profile: Preventing catastrophic pandemics↗🔗 web★★★☆☆80,000 HoursProblem profile: Preventing catastrophic pandemicsThis 80,000 Hours problem profile is a key reference for the effective altruism community's prioritization of engineered pandemic risk, complementing AI safety as a top concern and guiding career decisions in biosecurity.80,000 Hours' problem profile on catastrophic pandemic prevention, focusing primarily on engineered pandemics as an existential risk. It argues this is one of the world's most p...existential-riskbiosecuritygovernancepolicy+4Source ↗
• Bulletin of the Atomic Scientists (2024): Could AI help bioterrorists unleash a new pandemic?↗🔗 webCould AI help bioterrorists unleash a new pandemic?A 2024 Bulletin of the Atomic Scientists piece summarizing empirical research on AI biosecurity uplift risk; relevant to debates about AI capability thresholds, deployment safeguards, and biosecurity governance.This Bulletin of the Atomic Scientists article covers research examining whether current AI systems provide meaningful 'uplift' to would-be bioterrorists seeking to create or de...biosecuritydual-use-researchexistential-riskcapabilities+5Source ↗
• Undark (2024): The Long, Contentious Battle to Regulate Gain-of-Function Work↗🔗 webThe Long, Contentious Battle to Regulate Gain-of-Function WorkRelevant background for AI safety researchers studying dual-use research governance, as analogous regulatory challenges around dangerous capabilities research may inform how AI development oversight frameworks are designed and enforced.This article traces the decades-long regulatory struggle over gain-of-function (GOF) research, examining how scientists, policymakers, and biosecurity experts have clashed over ...governancebiosecuritydual-use-researchexistential-risk+3Source ↗
• Science (2025): NIH suspends dozens of pathogen studies over 'gain-of-function' concerns↗📄 paper★★★★★Science (peer-reviewed)Executive order blockedCovers NIH's suspension of pathogen research studies in response to executive order on gain-of-function oversight, relevant to AI safety's biosecurity concerns and dual-use research governance frameworks.Lloyd S. Etheredge (1985)In response to a Trump executive order on gain-of-function (GOF) research oversight, the National Institutes of Health (NIH) has suspended dozens of federally-funded pathogen st...biosecuritydual-use-researchx-riskSource ↗

Video & Audio

• 80,000 Hours Podcast: Kevin Esvelt on Biosecurity↗🔗 web★★★☆☆80,000 Hours80,000 Hours: Toby Ord on The PrecipiceA long-running podcast from the 80,000 Hours career advice organization; widely listened to in the EA and AI safety communities as a source of accessible, substantive conversations with key researchers and thinkers.The 80,000 Hours Podcast hosts in-depth interviews with leading researchers and thinkers on AI safety, existential risk, effective altruism, and related high-impact topics. It c...ai-safetyalignmentexistential-riskgovernance+5Source ↗ - MIT researcher on biological risks and pandemic preparedness
• Lex Fridman #431: Roman Yampolskiy↗🔗 webRoman Yampolskiy: Dangers of Superintelligent AI | Lex Fridman Podcast #431A long-form podcast interview with Roman Yampolskiy, a prominent pessimist voice in AI safety, offering accessible discussion of core control and alignment problems for a general audience.Lex Fridman interviews AI safety researcher Roman Yampolskiy about the existential risks of AGI and superintelligent AI, covering topics from AI controllability and deception to...ai-safetyexistential-riskalignmenttechnical-safety+4Source ↗ - Discusses AI safety including CBRN risks
• Future of Life Institute: Podcast series↗🔗 web★★★☆☆Future of Life InstituteFuture of Life Institute: Existential Risk PodcastsFLI's podcast is a long-running series hosted by figures like Lex Fridman (early episodes) and Lucas Perry, providing accessible overviews of x-risk topics; useful for newcomers or those seeking expert perspectives in audio format.The Future of Life Institute podcast series features conversations with leading researchers, policymakers, and thinkers on existential risks including AI safety, biosecurity, nu...existential-riskai-safetygovernancebiosecurity+4Source ↗ - Multiple episodes on biosecurity
• RAND: The AI and Biological Weapons Threat↗🔗 web★★★★☆RAND CorporationThe AI and Biological Weapons ThreatA 2023 RAND empirical study directly relevant to catastrophic risk from AI misuse; provides early evidence on LLM dual-use risks in bioweapons contexts, informing debates about frontier model deployment safeguards and biosecurity policy.This RAND Corporation report examines the misuse risks of large language models (LLMs) in biological weapons development through a red-team methodology. Preliminary findings sho...biosecurityred-teamingcapabilitiesexistential-risk+6Source ↗ - Video briefing on the 2024 study

Analytical Models