Founders Pledge - Building Global Resilience to AI Risks

Building global resilience to AI risks Many philanthropists interested in reducing AI-related risks have focused on interventions that could help prevent the deployment of unsafe AI. Fewer funders are investing in resilience : defenses that help prepare us for scenarios after deployment, with the goal of preventing advanced AI from becoming an existential threat.

 Funding opportunities that boost global resilience to AI risks, such as by developing better warning systems and incident response plans, can help fill a critical gap in our defenses. We recently made a grant from our Global Catastrophic Risks Fund that focuses on improving how the US government detects and responds to advanced AI threats , with the goal of building resilience and ensuring more robust AI safety policy.

 In this blog post, we’ll:

 
 Discuss the role building resilience plays in a “defense in depth” framework

 Explore the ways we can invest in global resilience against AI risks

 Explain the goals of our recent grant and the unique impact of the GCR Fund in this space

 
 The defense in depth framework for AI risks

 Imagine developing a strategy to protect a building from fire hazards. You might start by setting up preventative safety protocols to make a fire unlikely to happen, but that probably wouldn’t be the only step you would take. You’d also want to set up a smoke alarm system to warn you as early as possible when a fire begins, make sure fire extinguishers are easily accessible, and create an evacuation plan to minimize damage in the worst-case scenario.

 This type of risk management approach, which involves using multiple layers of protection to create a reliable safety system, is called a “ defense in depth ” framework. This type of multi-layered framework is used in many fields, including nuclear safety and information security . The goal is to make sure that the system remains resilient even if any one layer of protection fails.

 The “defense in depth” approach also applies to potentially existential risks, like misaligned AI. We recently published a deep-dive research report about the existential threats that could emerge from transformative AI, ranging from power-seeking AI systems to the misuse of AI for biological weapons. When it comes to mitigating AI risks and other extinction-level risks , a defense in depth framework involves three “defensive layers”:

 
 Prevention : Reducing the chance that a local catastrophe occurs, i.e., a dangerous AI system being deployed.

 Response : Reducing the likelihood that the catastrophe becomes global, i.e., a dangerous AI system gaining widespread power.

 Resilience : Reducing the likelihood that the global catastrophe leads to an existential catastrophe, i.e., a dangerous AI system threatening the destruction of humanity’s long-term potential.

 
 Currently, a large proportion of funding and attention related to AI risk mitigation focuses on the prevention component, but all three layers are needed for a rob

... (truncated, 9 KB total)