AI Safety Organizations (Overview)

Overview

The AI safety organizational landscape spans dedicated alignment research labs, policy think tanks, advocacy groups, and field-building institutions. These organizations aim to reduce catastrophic and existential risks from advanced AI systems through technical research, governance advocacy, talent development, and public engagement.

Funding is heavily concentrated through a small number of major funders, most prominently Coefficient GivingOrganizationCoefficient GivingCoefficient Giving (formerly Open Philanthropy) has directed $4B+ in grants since 2014, including $336M to AI safety (~60% of external funding). The organization spent ~$50M on AI safety in 2024, w...Quality: 55/100, which has provided grants to the majority of organizations listed on this page. This concentration produces a relatively coordinated funding environment, with most grantees sharing compatible research agendas and norms, while also reducing diversification of funding sources across the field.

Note: The AI safety organizational landscape evolves rapidly. Headcount, budget, and focus area descriptions reflect available information as of mid-2025 and may not capture recent changes. Check individual entity pages for the most current details.

Alignment Research Labs

Dedicated organizations conducting technical AI safety research:

• ARC (Alignment Research Center)OrganizationAlignment Research CenterComprehensive reference page on ARC (Alignment Research Center), covering its evolution from a dual theory/evals organization to ARC Theory (3 permanent researchers) plus the METR spin-out (Decembe...Quality: 57/100: Founded by Paul ChristianoPersonPaul ChristianoComprehensive biography of Paul Christiano documenting his technical contributions (IDA, debate, scalable oversight), risk assessment (~10-20% P(doom), AGI 2030s-2040s), and evolution from higher o...Quality: 39/100; focuses on alignment evaluation and theoretical alignment research
• METROrganizationMETRMETR conducts pre-deployment dangerous capability evaluations for frontier AI labs (OpenAI, Anthropic, Google DeepMind), testing autonomous replication, cybersecurity, CBRN, and manipulation capabi...Quality: 66/100: Evaluates dangerous capabilities in frontier AI models; spun out of ARC
• Apollo ResearchOrganizationApollo ResearchApollo Research demonstrated in December 2024 that all six tested frontier models (including o1, Claude 3.5 Sonnet, Gemini 1.5 Pro) engage in scheming behaviors, with o1 maintaining deception in ov...Quality: 58/100: Focuses on detecting and understanding deceptive AI behavior, including schemingRiskSchemingScheming—strategic AI deception during training—has transitioned from theoretical concern to observed behavior across all major frontier models (o1: 37% alignment faking, Claude: 14% harmful compli...Quality: 74/100 evaluations
• Redwood ResearchOrganizationRedwood ResearchA nonprofit AI safety and security research organization founded in 2021, known for pioneering AI Control research, developing causal scrubbing interpretability methods, and conducting landmark ali...Quality: 78/100: Alignment research lab working on interpretabilityResearch AreaInterpretabilityMechanistic interpretability has extracted 34M+ interpretable features from Claude 3 Sonnet with 90% automated labeling accuracy and demonstrated 75-85% success in causal validation, though less th...Quality: 66/100, adversarial trainingApproachAdversarial TrainingAdversarial training, universally adopted at frontier labs with $10-150M/year investment, improves robustness to known attacks but creates an arms race dynamic and provides no protection against mo...Quality: 58/100, and AI controlResearch AreaAI ControlAI Control is a defensive safety approach that maintains control over potentially misaligned AI through monitoring, containment, and redundancy, offering 40-60% catastrophic risk reduction if align...Quality: 75/100
• ConjectureOrganizationConjectureConjecture is a 30-40 person London-based AI safety org founded 2022, pursuing Cognitive Emulation (CoEm) - building interpretable AI from ground-up rather than aligning LLMs - with $30M+ Series A ...Quality: 37/100: Alignment research and product company based in London
• FAR AIOrganizationFAR AIFAR AI is an AI safety research nonprofit founded in July 2022 by Adam Gleave (CEO) and Karl Berzins (Co-founder & President). Based in Berkeley, California, the organization conducts technical res...Quality: 76/100: Researches robustness, adversarial attacks, and alignment failures in AI systems
• Palisade ResearchOrganizationPalisade ResearchPalisade Research is a 2023-founded nonprofit conducting empirical research on AI shutdown resistance and autonomous hacking capabilities, with notable findings that some frontier models resist shu...Quality: 65/100: Focuses on practical AI safety evaluation and red-teaming
• Seldon LabOrganizationSeldon LabSeldon Lab is a San Francisco-based AI safety accelerator founded in early 2025 that combines research publication with startup investment, claiming early success with portfolio companies raising $...Quality: 45/100: Works on alignment approaches and safety evaluations
• GoodfireOrganizationGoodfireGoodfire is a well-funded AI interpretability startup valued at $1.25B (Feb 2026) developing mechanistic interpretability tools like Ember API to make neural networks more transparent and steerable...Quality: 68/100: Interpretability-focused startup building tools for understanding neural networks
• MIRI (Machine Intelligence Research Institute)OrganizationMachine Intelligence Research InstituteComprehensive organizational history documenting MIRI's trajectory from pioneering AI safety research (2000-2020) to policy advocacy after acknowledging research failure, with detailed financial da...Quality: 50/100: Pioneer in AI alignmentApproachAI AlignmentComprehensive review of AI alignment approaches finding current methods (RLHF, Constitutional AI) show 75%+ effectiveness on measurable safety metrics for existing systems but face critical scalabi...Quality: 91/100 theory; founded 2000

Policy and Governance Organizations

Think tanks and research centers focused on AI governance and policy:

• GovAIOrganizationGovAIGovAI is an AI policy research organization with ~40-45 staff, funded primarily by Coefficient Giving ($1.8M+ in 2023-2024), that has trained 100+ governance researchers through fellowships and cur...Quality: 43/100: Research center focused on AI governance based at Oxford
• CSET (Center for Security and Emerging Technology)OrganizationGeorgetown CSETCSET is a $100M+ Georgetown center with 50+ staff conducting data-driven AI policy research, particularly on U.S.-China competition and export controls. The center conducts hundreds of annual gover...Quality: 43/100: Georgetown think tank producing policy-relevant research on AI and emerging technologies
• CSER (Centre for the Study of Existential Risk)OrganizationCSER (Centre for the Study of Existential Risk)CSER is a Cambridge-based existential risk research centre founded in 2012, now funded at ~$1M+ annually from FLI and other sources, producing 24+ publications in 2022 across AI safety, biosecurity...Quality: 58/100: Cambridge-based research center studying existential risks including from AI
• Secure AI ProjectOrganizationSecure AI ProjectPolicy advocacy organization founded ~2022-2023 by Nick Beckstead focusing on legislative requirements for AI safety protocols, whistleblower protections, and risk mitigation incentives. Rated high...Quality: 47/100: Advocacy organization focused on AI safety policy
• ControlAIOrganizationControlAIControlAI is a UK-based advocacy organization that has achieved notable policy engagement success (briefing 150+ lawmakers, securing support from 100+ UK parliamentarians) while promoting direct in...Quality: 63/100: Advocacy organization pushing for stronger AI regulation and safety standards
• Pause AIOrganizationPause AIPause AI is a grassroots advocacy movement founded May 2023 calling for international pause on frontier AI development until safety proven, growing to multi-continental network but achieving zero d...Quality: 59/100: Grassroots advocacy movement calling for a pause on frontier AI development
• Frontier Model ForumOrganizationFrontier Model ForumThe Frontier Model Forum represents the AI industry's primary self-governance initiative for frontier AI safety, establishing frameworks and funding research, but faces fundamental criticisms about...Quality: 58/100: Industry-led consortium for frontier AI safety, founded by AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials ($380B valuation, $14B ARR at Series G growing to $19B by March 2026), safety research (Constitutional AI, mechanistic interpretability...Quality: 74/100, Google DeepMindOrganizationGoogle DeepMindComprehensive overview of DeepMind's history, achievements (AlphaGo, AlphaFold with 200M+ protein structures), and 2023 merger with Google Brain. Documents racing dynamics with OpenAI and new Front...Quality: 37/100, Microsoft, and OpenAIOrganizationOpenAIComprehensive organizational profile of OpenAI documenting evolution from 2015 non-profit to Public Benefit Corporation, with detailed analysis of governance crisis, 2024-2025 ownership restructuri...Quality: 62/100. The forum's stated mission centers on safety research and best-practice sharing; observers differ on the extent to which it functions as a coordination body versus an industry advocacy vehicle
• IAPS (Institute for AI Policy and Strategy)OrganizationInstitute for AI Policy and StrategyAI policy think tank founded in 2024, headquartered in the Washington, DC metro area. Led by Jennifer Marron (Executive Director), Brianna Rosen (Research Director, Frontier Security), Amanda El-Da...: Nonpartisan think tank producing policy research from today's advanced models to potential AGI and superintelligence, and a significant talent pipeline into government
• Partnership on AIOrganizationPartnership on AIPartnership on AI (PAI) is a multi-stakeholder body founded in 2016 by Amazon, Apple, DeepMind, Google, IBM, Meta, and Microsoft. With 100+ partners, it develops shared norms for responsible AI acr...: Multi-stakeholder nonprofit founded by major tech companies with 100+ partners, setting shared norms for responsible AI across industry and civil society
• Ada Lovelace InstituteOrganizationAda Lovelace InstituteThe Ada Lovelace Institute is a UK-based independent research body funded by the Nuffield Foundation that bridges technical AI research with human rights-focused policymaking. It has produced influ...: Independent UK research institute (Nuffield Foundation) bridging technical AI research with human rights-focused policymaking
• AI Now InstituteOrganizationAI Now InstituteAI Now Institute at NYU is a pioneer in researching the social and political implications of AI. Founded by Meredith Whittaker and Kate Crawford in 2017, it produces influential annual reports on A...: NYU-based institute pioneering research on algorithmic bias, surveillance, and corporate AI power concentration
• AI Policy InstituteOrganizationAI Policy InstituteThe AI Policy Institute (AIPI) is a DC-based advocacy organization founded in 2023 that focuses on channeling public concern about AI into effective policy. Known for extensive public opinion polli...: DC-based advocacy organization translating public concern about AI into effective governance
• Americans for Responsible InnovationOrganizationAmericans for Responsible InnovationAmericans for Responsible Innovation (ARI) is a DC-based AI policy group founded in 2023 with backing from EA-aligned donors including Dustin Moskovitz's Open Philanthropy network. ARI focuses on b...: DC-based policy group focused on bipartisan AI safety legislation, backed by EA-aligned donors
• The Future SocietyOrganizationThe Future SocietyThe Future Society is an international nonprofit founded in 2014 at Harvard Kennedy School that works on AI governance across the UN, OECD, EU, and G7. Known for convening 44+ civil society organiz...: International nonprofit working on AI governance across the UN, OECD, EU, and G7

Major Think Tank AI Programs

Established policy research institutions with significant AI programs:

• Brookings AIET InitiativeOrganizationBrookings Institution AI and Emerging Technology InitiativeThe Brookings AIET Initiative is one of the most-cited think tank programs on AI policy in Washington. Part of the Governance Studies program, it produces influential research on AI regulation, wor...: One of the most-cited think tank programs on AI governance, workforce impacts, and algorithmic accountability
• RAND CorporationOrganizationRAND Corporation AI Policy ResearchRAND Corporation's AI policy research shapes Pentagon and NATO thinking on autonomous weapons, escalation risk, and AI-enabled warfare. RAND's AI work spans national security, defense applications,...: AI policy research shaping Pentagon and NATO thinking on autonomous weapons and escalation risk
• Stanford HAIOrganizationStanford Institute for Human-Centered Artificial Intelligence (HAI)Stanford HAI is an interdisciplinary institute with 200+ affiliated faculty that produces the widely cited AI Index Report tracking global AI trends. Founded in 2019 by Fei-Fei Li and John Etchemen...: Interdisciplinary institute producing the widely-cited AI Index Report, with 200+ affiliated faculty
• Carnegie Endowment AI ProgramOrganizationCarnegie Endowment for International Peace AI ProgramCarnegie's AI program researches how AI reshapes global governance, geopolitics, and democratic institutions. Operating through offices in Washington, Beijing, Brussels, Beirut, and New Delhi, Carn...: Researches AI's intersection with geopolitics and democratic institutions, with offices globally including Beijing
• CSIS Wadhwani CenterOrganizationCSIS Wadhwani Center for AI and Advanced TechnologiesThe Wadhwani Center for AI and Advanced Technologies at CSIS publishes influential research on AI in national security, military competition, autonomous weapons, and US-China tech rivalry. Establis...: Research on AI and national security, military competition, and US-China tech rivalry
• Center for Democracy and TechnologyOrganizationCenter for Democracy and TechnologyCDT is one of the oldest and most established digital rights organizations engaging on AI policy, founded in 1994. Its AI Governance Lab focuses on algorithmic accountability, automated decision-ma...: Digital rights nonprofit (est. 1994) with AI Governance Lab covering algorithmic accountability and workers' rights

Field-Building and Talent Development

Organizations supporting the growth of the AI safety field:

• 80,000 HoursOrganization80,000 Hours80,000 Hours is the largest EA career organization, reaching 10M+ readers and reporting 3,000+ significant career plan changes, with 80% of $10M+ funding from Coefficient Giving. Since 2016 they've...Quality: 45/100: Career advisory organization directing talent toward high-impact careers including AI safety
• MATS (ML Alignment Theory Scholars)OrganizationMATS ML Alignment Theory Scholars programMATS is a well-documented 12-week fellowship program that has successfully trained 213 AI safety researchers with strong career outcomes (80% in alignment work) and research impact (160+ publicatio...Quality: 60/100: Training program connecting aspiring alignment researchers with mentors
• Lightning Rod LabsOrganizationLightning Rod LabsLightning Rod Labs is an early-stage AI company using temporal data to train prediction models, claiming 10% returns on prediction markets but with limited independent validation. The company has n...Quality: 38/100: Works on AI safety infrastructure and tooling
• AI Futures ProjectOrganizationAI Futures ProjectAI Futures Project is a nonprofit co-founded in 2024 by Daniel Kokotajlo, Eli Lifland, and Thomas Larsen that produces detailed AI capability forecasts, most notably the AI 2027 scenario depicting ...Quality: 50/100: Research and analysis on AI development trajectories and safety considerations

Research and Analysis

Organizations focused on understanding AI progress and risks:

• Epoch AIOrganizationEpoch AIEpoch AI maintains comprehensive databases tracking 3,200+ ML models showing 4.4x annual compute growth and projects data exhaustion 2026-2032. Their empirical work directly informed EU AI Act's 10...Quality: 51/100: Tracks AI compute trends, model capabilities, and training data
• CAIS (Center for AI Safety)OrganizationCenter for AI SafetyCAIS is a nonprofit research organization founded by Dan Hendrycks that has distributed compute grants to researchers, published technical AI safety papers including the representation engineering ...Quality: 42/100: Conducts safety research and field-building for AI safety; hosts a compute cluster for safety research
• CHAI (Center for Human-Compatible AI)OrganizationCenter for Human-Compatible AICHAI is UC Berkeley's AI safety research center founded by Stuart Russell in 2016, pioneering cooperative inverse reinforcement learning and human-compatible AI frameworks. The center has trained 3...Quality: 37/100: UC Berkeley research center founded by Stuart RussellPersonStuart RussellStuart Russell (born 1962) is a British computer scientist and UC Berkeley professor who co-authored the dominant AI textbook 'Artificial Intelligence: A Modern Approach' (used in over 1,500 univer...Quality: 30/100 focusing on human-compatible AI

Budget and Headcount Comparison

For funders and researchers evaluating organizational capacity and capital efficiency, comparative budget and headcount data can help identify where additional resources may be most impactful and how different organizations structure their research operations.

The table below aggregates publicly available estimates across nine prominent independent AI safety organizations.

All figures are estimates derived from IRS Form 990 filings (via ProPublica Nonprofit Explorer), Coefficient Giving (formerly Open Philanthropy) grant disclosures, LinkedIn headcount data, and news reports. Figures are approximate, may lag actual values by one to two years, and should be treated as indicative rather than authoritative. The "Est. Budget per Staff Member/year" column is calculated using the midpoint of the headcount range and counts all staff, not researchers only.

The budget-per-staff figures reflect meaningful variation in organizational structure. Organizations with lower ratios (e.g., Conjecture) typically employ a higher proportion of non-researcher staff or operate hybrid research-product models, whereas those with higher ratios (e.g., MIRI) tend toward smaller, senior-heavy research teams. These figures should not be interpreted as proxies for research quality or output volume.

Key Patterns

Specialization trend: The field has moved from generalist safety organizations—such as MIRIOrganizationMachine Intelligence Research InstituteComprehensive organizational history documenting MIRI's trajectory from pioneering AI safety research (2000-2020) to policy advocacy after acknowledging research failure, with detailed financial da...Quality: 50/100 and the Future of Humanity InstituteOrganizationFuture of Humanity InstituteThe Future of Humanity Institute (2005-2024) was a pioneering Oxford research center that founded existential risk studies and AI alignment research, growing from 3 to ~50 researchers and receiving...Quality: 51/100 (FHI, which closed in 2024)—toward more specialized roles: dedicated evaluation labs (METROrganizationMETRMETR conducts pre-deployment dangerous capability evaluations for frontier AI labs (OpenAI, Anthropic, Google DeepMind), testing autonomous replication, cybersecurity, CBRN, and manipulation capabi...Quality: 66/100, Apollo ResearchOrganizationApollo ResearchApollo Research demonstrated in December 2024 that all six tested frontier models (including o1, Claude 3.5 Sonnet, Gemini 1.5 Pro) engage in scheming behaviors, with o1 maintaining deception in ov...Quality: 58/100), interpretability startups (GoodfireOrganizationGoodfireGoodfire is a well-funded AI interpretability startup valued at $1.25B (Feb 2026) developing mechanistic interpretability tools like Ember API to make neural networks more transparent and steerable...Quality: 68/100), policy research centers (ControlAIOrganizationControlAIControlAI is a UK-based advocacy organization that has achieved notable policy engagement success (briefing 150+ lawmakers, securing support from 100+ UK parliamentarians) while promoting direct in...Quality: 63/100, GovAIOrganizationGovAIGovAI is an AI policy research organization with ~40-45 staff, funded primarily by Coefficient Giving ($1.8M+ in 2023-2024), that has trained 100+ governance researchers through fellowships and cur...Quality: 43/100), and talent pipelines (MATSOrganizationMATS ML Alignment Theory Scholars programMATS is a well-documented 12-week fellowship program that has successfully trained 213 AI safety researchers with strong career outcomes (80% in alignment work) and research impact (160+ publicatio...Quality: 60/100, 80,000 HoursOrganization80,000 Hours80,000 Hours is the largest EA career organization, reaching 10M+ readers and reporting 3,000+ significant career plan changes, with 80% of $10M+ funding from Coefficient Giving. Since 2016 they've...Quality: 45/100).

Industry-adjacent positioning: Organizations in this landscape occupy a range of positions relative to frontier AI developers. Some—such as the Frontier Model ForumOrganizationFrontier Model ForumThe Frontier Model Forum represents the AI industry's primary self-governance initiative for frontier AI safety, establishing frameworks and funding research, but faces fundamental criticisms about...Quality: 58/100, Redwood ResearchOrganizationRedwood ResearchA nonprofit AI safety and security research organization founded in 2021, known for pioneering AI Control research, developing causal scrubbing interpretability methods, and conducting landmark ali...Quality: 78/100, and Apollo ResearchOrganizationApollo ResearchApollo Research demonstrated in December 2024 that all six tested frontier models (including o1, Claude 3.5 Sonnet, Gemini 1.5 Pro) engage in scheming behaviors, with o1 maintaining deception in ov...Quality: 58/100—maintain active collaborative relationships with frontier labs. Others, including Pause AIOrganizationPause AIPause AI is a grassroots advocacy movement founded May 2023 calling for international pause on frontier AI development until safety proven, growing to multi-continental network but achieving zero d...Quality: 59/100 and ControlAIOrganizationControlAIControlAI is a UK-based advocacy organization that has achieved notable policy engagement success (briefing 150+ lawmakers, securing support from 100+ UK parliamentarians) while promoting direct in...Quality: 63/100, advocate for regulatory constraints on AI development and position themselves independently of industry partnerships. Proponents of each approach offer different accounts of how safety outcomes are best achieved.

Funding concentration: As illustrated in the budget table above, most organizations in this cluster report Coefficient GivingOrganizationCoefficient GivingCoefficient Giving (formerly Open Philanthropy) has directed $4B+ in grants since 2014, including $336M to AI safety (~60% of external funding). The organization spent ~$50M on AI safety in 2024, w...Quality: 55/100 as their primary funder. This pattern is visible across alignment research, governance research, and field-building organizations alike.