Back
Building an early warning system for LLM-aided biological threat creation
webCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: OpenAI
An OpenAI empirical study and methodological blueprint for assessing LLM-enabled bioweapon risk, directly tied to their Preparedness Framework; represents one of the first systematic attempts to quantify AI uplift in biological threat scenarios.
Metadata
Importance: 72/100blog postprimary source
Summary
OpenAI presents a methodology for evaluating whether LLMs like GPT-4 could meaningfully assist malicious actors in creating biological threats. In a controlled study with 100 participants (50 PhD biology experts, 50 students), they found GPT-4 provides at most mild uplift in biological threat creation accuracy compared to internet-baseline resources. The work is framed as a blueprint for empirical biosecurity evaluation and a potential 'tripwire' for future capability monitoring.
Key Points
- •GPT-4 provided only mild, non-conclusive uplift in biological threat creation accuracy compared to existing internet resources as a baseline.
- •Study involved 100 participants: 50 PhD-level biology experts with wet lab experience and 50 student-level participants, comparing AI-assisted vs. unassisted conditions.
- •Introduces a replicable evaluation blueprint intended as a 'tripwire' signal for when models may require heightened caution around bioweapons misuse.
- •Connected to OpenAI's Preparedness Framework, which tracks catastrophic risk categories including CBRN (chemical, biological, radiological, nuclear) threats.
- •Calls for community input and methods-sharing to improve evaluation rigor, acknowledging this as early-stage work with limitations.
Cited by 2 pages
| Page | Type | Quality |
|---|---|---|
| AI Uplift Assessment Model | Analysis | 70.0 |
| Bioweapons Risk | Risk | 91.0 |
Cached Content Preview
HTTP 200Fetched Mar 20, 202648 KB
OpenAI
January 31, 2024
[Publication](https://openai.com/research/index/publication/)
# Building an early warning system for LLM-aided biological threat creation
[View data](https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/#_4SUgyekFkwzOK2Vq8ES0pq)
Loading…
Share
We’re developing a blueprint for evaluating the risk that a large language model (LLM) could aid someone in creating a biological threat.
In an evaluation involving both biology experts and students, we found that GPT‑4 provides at most a mild uplift in biological threat creation accuracy. While this uplift is not large enough to be conclusive, our finding is a starting point for continued research and community deliberation.
## Overview
_Note: As part of our_ [_Preparedness Framework_ ](https://openai.com/preparedness/) _, we are investing in the development of improved evaluation methods for AI-enabled safety risks. We believe that these efforts would benefit from broader input, and that methods-sharing could also be of value to the AI risk research community. To this end, we are presenting some of our early work—today, focused on biological risk. We look forward to community feedback, and to sharing more of our ongoing research._
**Background.** As OpenAI and other model developers build more capable AI systems, the potential for both beneficial and harmful uses of AI will grow. One potentially harmful use, highlighted by researchers and policymakers, is the ability for AI systems to assist malicious actors in creating biological threats (e.g., see [White House 2023(opens in a new window)](https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/), [Lovelace 2022(opens in a new window)](https://www.washingtontimes.com/news/2022/sep/12/ai-powered-biological-warfare-biggest-issue-former/), [Sandbrink 2023(opens in a new window)](https://www.vox.com/future-perfect/23820331/chatgpt-bioterrorism-bioweapons-artificial-inteligence-openai-terrorism)). In one discussed hypothetical example, a malicious actor might use a highly-capable model to develop a step-by-step protocol, troubleshoot wet-lab procedures, or even autonomously execute steps of the biothreat creation process when given access to tools like [cloud labs(opens in a new window)](https://www.theguardian.com/science/2022/sep/11/cloud-labs-and-remote-research-arent-the-future-of-science-theyre-here) (see [Carter et al., 2023(opens in a new window)](https://www.nti.org/analysis/articles/the-convergence-of-artificial-intelligence-and-the-life-sciences
... (truncated, 48 KB total)Resource ID:
2f918741de446a84 | Stable ID: MDMxZmM2ZT