Controlled Vocabulary for Longtermist Analysis

Executive Summary

Terminology System

Risk Domains

The primary categories of existential and catastrophic risk:

Cross-Cutting Modifiers

Modifiers can be appended to any domain when the distinction matters:

Composition Examples

Bio                           # General biological risk
Bio-Engineered                # When engineering is relevant
Bio-Engineered-Deliberate     # True bioweapon scenario
Bio-State-Offensive           # State bioweapons capability

Cyber-Individual              # Lone actor cyber threat
NUC-State-Accidental          # Nuclear accident by state actor
EPIS-Corp-Deliberate          # Corporate disinformation campaign

AI Stages (Context Modifier)

AI stages describe when risks and interventions occur relative to transformative AI. These are abstract by default, with optional concrete annotations.

Concrete Annotations (Optional)

When making specific claims, annotate with your assumptions:

Near (~2028)
Mid (TAI-50% capability)
Mid (10^28 FLOP threshold)
TAI (assuming 2035 timeline)

The abstract terms are the controlled vocabulary. Concrete annotations clarify individual assumptions without forcing agreement on timelines.

Combining AI Stage with Risk Domains

Every non-AI risk domain gets transformed by AI stage:

Bio-Near                  # Bio risks in current context
Bio-Mid                   # Bio risks as AI advances
Bio-TAI                   # Bio risks during transformation
Cyber-Mid-State           # State cyber threats in mid-period
NUC-TAI-Accidental        # Nuclear accident risk during TAI transition

State Variables

Concepts for describing the current risk landscape:

Example usage:

Bio-Near-Offensive-Potential     # Current bioweapon capability
Bio-Near-Defensive-Potential     # Current biodefense capability
Bio-Near-Exposure                # Net bio vulnerability today

Intervention Categories

By Mechanism

By Target

Composition

Technical-Bio-Prevention         # Technical solutions to prevent biopandemics
Governance-MAI-Detection         # Governance for detecting AI misalignment
Coordination-NUC-Mitigation      # International coordination to limit nuclear damage

Epistemic Markers

For qualifying claims and tracking disagreements:

Visual Presentation

In diagrams and interfaces, vocabulary terms appear as colored clickable tags:

Tag Display

┌─────────────────────────────────────────────────────┐
│ AI-enabled pandemic creation by non-state actors   │
│                                                     │
│ [Bio] [Engineered] [Mid] [Individual] [Deliberate] │
│  red    orange    blue    purple       yellow      │
└─────────────────────────────────────────────────────┘

Each tag:

• Has a consistent color by category (domains = red, AI stages = blue, etc.)
• Is clickable to filter/search for related concepts
• Expands on hover to show definition

Alternative Layouts

For different contexts:

Inline with separators:

Bio · Engineered · Mid · Individual

Primary + context:

┌─────────────────────────────────┐
│ Bio Risk                        │  ← Primary (large)
│ in Mid period, Individual actor │  ← Context (smaller)
└─────────────────────────────────┘

Use Cases

1. Tagging Concrete Risks

When discussing specific risks, apply vocabulary tags for consistency:

This enables:

• Search: Find all Bio-Mid risks
• Comparison: Compare State vs Individual actor risks
• Gap analysis: Which combinations have no coverage?

2. Combinatorial Risk Maps

Generate importance scores across all meaningful combinations:

Simple 2D Slice: Domain × AI Stage

Scores are illustrative. Actual scores would be developed through structured elicitation.

Multi-Dimensional Analysis

Full dimensionality:

Domain (8) × AI-Stage (4) × Actor (6) × Intent (3) × Scale (4)
= 2,304 cells

With pruning of implausible combinations (e.g., NUC-Individual-Existential), perhaps 500-1000 meaningful cells.

What This Enables

1. Gap analysis: High importance + low attention = opportunity
2. Priority disputes made explicit: "You rate MAI-TAI as 10, I rate it 6"
3. Neglectedness identification: Which cells have no interventions?
4. Interaction effects: How does moving along one dimension shift others?

Open Questions

Granularity Decisions

• Should Cyber split into Cyber-Infrastructure, Cyber-Financial, Cyber-Military?
• Is Power-Concentrating-AI distinct enough from Authoritarian?
• Do we need AI-Accident separate from MAI?

Scoring Approach

Starting simple (single importance score), expanding over time:

• Phase 1: Single importance score (1-10)
• Phase 2: Probability × Severity decomposition
• Phase 3: Add Neglectedness, Tractability
• Phase 4: Full ITN framework per cell

Timeline Uncertainty

The AI stages are relative to TAI. How do we handle:

• People with very different TAI timelines?
• Possibility that TAI never arrives?
• Multiple transformation points rather than single TAI?

Positive Outcomes

Current vocabulary is risk-focused. Should we add:

• Flourishing-Potential
• Coordination-Success
• Alignment-Success

Implementation Notes

Data Model

type RiskConcept = {
  domain: Domain;                    // Bio, Cyber, NUC, etc.
  aiStage?: AIStage;                 // Near, Mid, TAI, Post-TAI
  modifiers?: Modifier[];            // Engineered, State, Deliberate, etc.
  stageAnnotation?: string;          // Optional: "~2028", "TAI-50%"
  importanceScore?: number;          // 1-10
  epistemicStatus?: EpistemicMarker; // Assumption, Crux, Contested, etc.
}

Serialization Format

For plain text contexts (filenames, URLs, search):

[Domain](-[Modifier])*(-[AIStage])?

Examples:
  bio-engineered-mid
  nuc-state-accidental-tai
  mai-existential-post-tai

Sources & References

AI Development Frameworks

• DeepMind Levels of AGI Paper - Capability-based AGI levels
• AGI Definitions Comparison - OpenAI, Anthropic, DeepMind approaches

Existential Risk Literature

• How Do AI Timelines Affect Existential Risk? - LessWrong analysis
• Existential Risk from AI - Wikipedia

Related Vocabulary Projects

• EA Forum: Crucial Consideration - Bostrom's framework
• CFAR Double Crux - Crux terminology origin