Pause / Moratorium

Overview

Pause and moratorium proposals represent the most direct governance intervention for AI safety: deliberately slowing or halting frontier AI development to allow safety research, governance frameworks, and societal preparation to catch up with rapidly advancing capabilities. These proposals range from targeted pauses on specific capability thresholds to comprehensive moratoria on all advanced AI development, with proponents arguing that the current pace of development may be outstripping humanity's ability to ensure safe deployment.

The most prominent call for a pause came in March 2023, when the Future of Life InstituteOrganizationFuture of Life InstituteComprehensive profile of FLI documenting $25M+ in grants distributed (2015: $7M to 37 projects, 2021: $25M program), major public campaigns (Asilomar Principles with 5,700+ signatories, 2023 Pause ...Quality: 46/100 (FLI) published an open letter calling for a six-month pause on training AI systems more powerful than GPT-4. Released just one week after GPT-4's launch, the letter garnered over 30,000 signatures, including prominent AI researchers such as Yoshua Bengio and Stuart RussellPersonStuart RussellStuart Russell (born 1962) is a British computer scientist and UC Berkeley professor who co-authored the dominant AI textbook 'Artificial Intelligence: A Modern Approach' (used in over 1,500 univer...Quality: 30/100, as well as technology leaders like Elon MuskPersonElon MuskComprehensive profile of Elon Musk's role in AI, documenting his early safety warnings (2014-2017), OpenAI founding and contentious departure, xAI launch and funding history, Neuralink BCI developm...Quality: 38/100 and Steve Wozniak. The letter cited risks including AI-generated propaganda, extreme automation of jobs, and a society-wide loss of control. However, no major AI laboratory implemented a voluntary pause, and the letter's six-month timeline passed without meaningful slowdown in frontier development. As MIT Technology Review noted six months later, AI companies instead directed "vast investments in infrastructure to train ever-more giant AI systems."

The fundamental logic behind pause proposals is straightforward: if AI development is proceeding faster than our ability to make it safe, slowing development provides time for safety work. As Bengio et al. wrote in Science in May 2024, "downside artificial intelligence risks must be managed effectively and urgently if posited AI benefits are to be realized safely." However, implementation faces severe challenges including competitive dynamics between nations and companies, enforcement difficulties, and concerns that pauses might push development underground or to jurisdictions with fewer safety constraints. These proposals remain controversial even within the AI safety community, with some arguing they are essential for survival and others viewing them as impractical or counterproductive.

Risk Assessment & Impact

Core Arguments

Case for Pausing

flowchart TD
  SPEED[AI Development Speed] --> GAP{Safety Gap?}
  SAFETY[Safety Research Speed] --> GAP

  GAP -->|Growing| RISK[Increasing Risk]
  GAP -->|Stable| MANAGE[Manageable]
  GAP -->|Shrinking| GOOD[Safety Catching Up]

  RISK --> OPTION1[Continue Development]
  RISK --> OPTION2[Pause Development]

  OPTION1 --> RACE[Racing to Danger]
  OPTION2 --> TIME[Buy Time for Safety]

  TIME --> RESEARCH[More Safety Research]
  TIME --> GOV[Better Governance]
  TIME --> PREP[Societal Preparation]

  RESEARCH --> SAFER[Safer Development]
  GOV --> SAFER
  PREP --> SAFER

  style RISK fill:#ffcccc
  style SAFER fill:#d4edda
  style RACE fill:#ff9999

Case Against Pausing

Pause Proposals Analyzed

FLI Open Letter (2023)

Notable critiques: AI researcher Andrew NgPersonAndrew NgAndrew Ng is one of the most prominent figures in AI education and research. He co-founded Google Brain, served as Chief Scientist at Baidu, and co-founded Coursera. His online courses have taught ... argued that "there is no realistic way to implement a moratorium" without government intervention, which would be "anti-competitive" and "awful innovation policy." Reid Hoffman criticized the letter as "virtue signaling" that would hurt the cause by alienating the AI developer community needed to achieve safety goals.

PauseAI Movement

Academic Proposals

Implementation Challenges

Coordination Problems

Enforcement Mechanisms

Unintended Consequences

Historical Precedents

Lessons for AI Pauses

• Narrow scope is more enforceable than broad moratoria
• Verification mechanisms are essential for compliance
• International coordination requires identifying mutual interests
• Community buy-in from researchers enables voluntary compliance
• Clear triggering conditions help define when restrictions apply

Scalability Assessment

Strategic Considerations

When Pauses Might Work

Alternative Approaches

Quick Assessment

Risks Addressed

If implemented effectively, pause/moratorium would address:

Limitations

• Enforcement Infeasibility: No mechanism to enforce global compliance
• Competitive Dynamics: Unilateral pause disadvantages safety-conscious actors
• Displacement Risk: Development may move to less cautious jurisdictions
• Definition Challenges: Unclear what should be paused
• Political Unreality: Insufficient political will for meaningful implementation
• Temporary Nature: Pauses must eventually end; doesn't solve underlying problem

International Coordination Efforts

While a full pause has not been achieved, international efforts toward AI governance have accelerated since the 2023 open letter:

flowchart TD
  LETTER[FLI Open Letter<br/>March 2023] --> BLETCHLEY[UK AI Safety Summit<br/>Bletchley Park Nov 2023]
  BLETCHLEY --> REPORT[International AI<br/>Safety Report Mandate]
  BLETCHLEY --> SEOUL[Seoul AI Safety Summit<br/>May 2024]

  REPORT --> BENGIO[Yoshua Bengio<br/>Chairs 30-Nation Panel]
  BENGIO --> INTERIM[Interim Report<br/>May 2024]
  INTERIM --> FINAL[Final Report<br/>Paris Feb 2025]

  SEOUL --> COMMITMENTS[Frontier AI Safety<br/>Commitments]
  SEOUL --> INSTITUTES[International Network<br/>of AI Safety Institutes]

  COMMITMENTS --> VOLUNTARY[16 Companies Sign<br/>Voluntary Commitments]
  INSTITUTES --> COLLAB[Formalized International<br/>Safety Collaboration]

  style LETTER fill:#e1f5fe
  style VOLUNTARY fill:#fff3e0
  style FINAL fill:#e8f5e9

Sources & Resources

Primary Sources

Key Organizations

Further Reading