Skip to content
Longterm Wiki
Navigation
Updated 2025-12-28HistoryData
Page StatusContent
Edited 3 months ago3.6k words62 backlinksUpdated every 3 weeksOverdue by 77 days
52QualityAdequate •32ImportanceReference48ResearchLow
Content7/13
SummaryScheduleEntityEdit historyOverview
Tables6/ ~14Diagrams1/ ~1Int. links32/ ~29Ext. links0/ ~18Footnotes0/ ~11References20/ ~11Quotes0Accuracy0RatingsN:2.5 R:5.5 A:4 C:6.5Backlinks62
Issues2
QualityRated 52 but structure suggests 73 (underrated by 21 points)
StaleLast edited 98 days ago - may need review

UK AI Safety Institute

Government

UK AI Safety Institute

The UK AI Safety Institute (renamed AI Security Institute in Feb 2025) operates with ~30 technical staff and 50M GBP annual budget, conducting frontier model evaluations using its open-source Inspect AI framework and coordinating the 10+ country International Network of AI Safety Institutes. April 2024 evaluations found frontier models capable of intermediate cybersecurity tasks and PhD-level biology knowledge, with safeguards vulnerable to basic jailbreaks.

TypeGovernment
Founded2023
LocationLondon, UK
Websitegov.uk
Related
People
Ian Hogarth
Organizations
US AI Safety InstituteMETRApollo ResearchAnthropicOpenAIGoogle DeepMind
Historicals
AI Safety Summit (Bletchley Park)
3.6k words · 62 backlinks

Overview

The UK AI Safety Institute (UK AISI) is a government organization established in November 2023 to advance AI safety through research, evaluation, and international coordination. Created in the wake of the first AI Safety Summit at Bletchley Park, AISI represents the UK's commitment to being a global leader in AI safety and governance. In February 2025, the organization was renamed the AI Security Institute to reflect a sharper focus on national security threats including cyberattacks, fraud, and AI-enabled bioweapon development.

As one of the first government bodies specifically dedicated to advanced AI safety (alongside the US AISI), the UK AISI plays a pioneering role in translating AI safety research into government policy and practice. The organization conducts technical research on AI risks, evaluates frontier models for dangerous capabilities, develops safety standards, and coordinates international efforts through the International Network of AI Safety Institutes, which now includes 10+ member countries.

The UK AISI's strategic positioning reflects the UK government's ambition: to be a global hub for AI safety, bridging technical research communities, frontier AI labs, and international policymakers. Former Chair Ian Hogarth described AISI as a "startup inside the government" that combines technical rigor with policy influence.

Organization at a Glance

AttributeDetails
FoundedNovember 2023 (following AI Safety Summit)
RenamedFebruary 2025 (to AI Security Institute)
Parent DepartmentDepartment for Science, Innovation and Technology (DSIT)
LeadershipIan Hogarth (former Chair, 2023-2025)
Technical Staff30+ researchers (as of mid-2024)
Annual BudgetApproximately 50 million GBP
LocationsLondon (HQ), San Francisco (opened 2024)
Key OutputInspect AI framework, Frontier AI Trends Report
International RoleFounding member of International Network of AI Safety Institutes

History and Founding

Background: UK AI Safety Summit (November 2023)

Bletchley Park Declaration:

  • UK hosted first international AI Safety Summit at Bletchley Park
  • 28 countries and EU signed declaration on AI risks
  • Recognized catastrophic risks from advanced AI
  • Committed to international cooperation on safety
  • Political moment crystallizing government attention

Summit outcomes:

  • Agreement on establishing AI Safety Institutes
  • UK and US announced creation of their institutes
  • Commitments from frontier AI labs
  • International research collaboration agreements
  • Foundation for ongoing coordination

Context:

  • ChatGPT moment (late 2022) raised AI awareness
  • Growing concern about AI existential risk
  • UK positioning for leadership role
  • Rishi Sunak government prioritizing AI policy
  • Academic and researcher advocacy for government action

Establishment (2023)

Founding: Announced at AI Safety Summit, operationalized late 2023

Organizational placement:

  • Part of UK Department for Science, Innovation and Technology (DSIT)
  • Government agency with civil service staff
  • Independent research capacity
  • Policy coordination role
  • International engagement mandate

Initial mission:

  • Conduct safety research on advanced AI
  • Evaluate frontier models for dangerous capabilities
  • Develop testing and evaluation methodologies
  • Coordinate international AI safety efforts
  • Advise UK government on AI risks and governance

Funding: The UK initially invested 100 million GBP in the Frontier AI Taskforce (AISI's predecessor), providing more funding for AI safety than any other country at the time. The Institute's annual budget is approximately 50 million GBP, continuing as an annual amount through the end of the decade, subject to demonstrating continued requirement. This makes the UK AISI an outlier in funding compared to other national AI Safety Institutes globally.

Early Development (2023-2024)

2023 activities: The Institute rapidly hired technical staff, recruiting senior alumni from OpenAI, Google DeepMind, and the University of Oxford. Ian Hogarth was appointed as Chair in June 2023 and served until 2025. By the end of 2023, the organization had established its organizational structure, built relationships with AI labs for model access, and begun coordination with the US AISI.

2024 milestones:

DateDevelopment
April 2024Published first evaluation results on publicly available frontier models
May 2024Open-sourced Inspect AI evaluation framework
May 2024Announced San Francisco office opening
May 2024AI Seoul Summit: formed International Network of AI Safety Institutes
November 2024First meeting of International Network in San Francisco
December 2024Published Frontier AI Trends Report

Current status (2025): Following the February 2025 renaming to AI Security Institute, the organization now focuses more explicitly on national security threats. Technology Secretary Peter Kyle announced the change at the Munich Security Conference, emphasizing a pivot from concerns about bias and misinformation toward risks such as cyberattacks, fraud, and AI-enabled bioweapon development. The Institute has launched a new criminal misuse team in partnership with the Home Office to investigate AI's role in serious crimes including child exploitation, cybercrime, and financial fraud.

Core Functions and Activities

1. Frontier Model Evaluations

The Institute independently assesses advanced AI systems for dangerous capabilities across domains critical to national security and public safety. In their April 2024 evaluation of publicly available frontier models, AISI found several concerning capabilities:

Evaluation DomainKey Findings (April 2024)
CybersecurityModels capable of completing basic to intermediate cybersecurity challenges
Chemistry/BiologySeveral models demonstrated PhD-equivalent level of knowledge
SafeguardsBuilt-in safeguards vulnerable to even basic "jailbreaks"
Autonomous SystemsModels performed better than biology PhD experts on open-ended questions

Methodology: The Institute has built evaluation expertise by learning from and collaborating with organizations like METR, Apollo Research, and ARC. Their approach includes both pre-deployment and post-deployment testing, with a focus on red-teaming and adversarial evaluation techniques.

Lab cooperation: AISI has established voluntary agreements with OpenAI, Anthropic, and Google DeepMind for pre-release access to models. Labs share safety-relevant information under NDAs, coordinate on safety thresholds, and collaborate on evaluation methodology development.

Inspect AI Framework: In May 2024, AISI open-sourced Inspect AI, a Python framework for building and running reproducible LLM evaluations. The framework has been adopted by major AI labs including Anthropic, DeepMind, and xAI. Inspect AI includes over 100 pre-built evaluations, a web-based visualization tool, and support for tool calling and multi-agent evaluations. Key benchmarks include ChemBench (2,786 chemistry questions), Humanity's Last Exam (3,000 questions across dozens of subjects), and Cybench (40 professional-level cybersecurity challenges).

2. Research Funding Programs

AISI manages substantial external research funding, making it one of the largest funders of AI safety research globally. The Institute's grant programs combine government funding with partnerships from industry and philanthropy.

ProgramTotal FundingPer-Project RangeFocus Areas
The Alignment Project15 million GBP50K-1M GBPAlignment research, interdisciplinary collaboration
Systemic Safety Grants8.5 million GBPUp to 200K GBPSocietal resilience, healthcare, energy grids, financial markets
Challenge Fund5 million GBPUp to 200K GBPSafeguards, control, alignment, societal resilience

The Alignment Project: This global fund is supported by an international coalition including the UK AISI, Canadian AI Safety Institute, Schmidt Sciences, Amazon Web Services, Halcyon Futures, Safe AI Fund, UKRI, Anthropic, and ARIA. Recipients can also receive up to 5 million GBP in dedicated AWS cloud computing credits, enabling technical experiments beyond typical academic reach.

Systemic Safety Grants: Launched in partnership with the Engineering and Physical Sciences Research Council (EPSRC) and Innovate UK (part of UKRI), the first phase allocated 4 million GBP to support approximately 20 projects. The program aims to increase societal resilience to widespread AI deployment across critical infrastructure.

3. Technical Safety Research

AISI conducts both in-house and collaborative research across several priority areas:

Technical research: Evaluation methodologies, interpretability and transparency techniques, robustness and reliability testing, alignment approaches, and measurement/benchmarking standards.

Sociotechnical research: AI governance frameworks, risk assessment methodologies, safety standards and best practices, international coordination mechanisms, and public engagement strategies.

Key outputs include: The Frontier AI Trends Report (December 2024), which provides an empirical assessment of frontier AI capabilities and their trajectory, and regular evaluation reports shared with policymakers and the public.

4. International Coordination

The UK has positioned itself as a convener for international AI safety efforts, hosting the first AI Safety Summit and co-founding the International Network of AI Safety Institutes.

Diagram (loading…)
flowchart TD
  UK[UK AI Security Institute] --> NETWORK[International Network of<br/>AI Safety Institutes]
  US[US AI Safety Institute] --> NETWORK
  NETWORK --> EU[European Commission]
  NETWORK --> JP[Japan]
  NETWORK --> KR[Republic of Korea]
  NETWORK --> SG[Singapore]
  NETWORK --> CA[Canada]
  NETWORK --> AU[Australia]
  NETWORK --> FR[France]
  NETWORK --> KE[Kenya]

  UK --> SF[San Francisco Office]
  SF -.->|Close collaboration| US

  style UK fill:#1e40af,color:#fff
  style NETWORK fill:#059669,color:#fff

International Network of AI Safety Institutes: During the AI Seoul Summit in May 2024, international leaders agreed to form this network comprising institutes from the UK, US, Japan, France, Germany, Italy, Singapore, South Korea, Australia, Canada, and the European Union. The first meeting was held in San Francisco in November 2024, with representatives from 10 countries and the European Commission.

US-UK partnership: The UK and US AISIs work as "sister organizations" with close collaboration on methodologies, coordinated evaluations, and joint international engagement. The San Francisco office (opened summer 2024) facilitates this partnership and allows AISI to recruit Bay Area talent and engage directly with major AI labs.

Seoul Statement goals: The international network aims to "accelerate the advancement of the science of AI safety" by promoting complementarity and interoperability between institutes and fostering a common international understanding of AI safety approaches.

5. Standard-Setting and Guidelines

Developing best practices:

Evaluation standards:

  • Methodologies for dangerous capability testing
  • Thresholds for deployment decisions
  • Red-teaming protocols
  • Transparency and documentation requirements
  • Continuous monitoring approaches

Safety standards:

  • Risk management frameworks
  • Safety assurance processes
  • Incident response and reporting
  • Governance and accountability
  • Alignment verification

UK and international adoption:

  • UK government procurement requirements
  • Industry adoption (voluntary initially)
  • International harmonization efforts
  • Foundation for potential regulation
  • Soft power and norm-setting

6. Policy Advice and Government Coordination

Advising UK government:

Policy development:

  • Risk assessments for ministers and parliament
  • Technical input to AI legislation
  • Regulatory framework design
  • International negotiation support
  • Public communication strategies

Interagency coordination:

  • National security agencies (GCHQ, MI5, MI6)
  • Regulatory bodies (ICO, CMA, Ofcom)
  • Research funders (UKRI)
  • Science and technology departments
  • Foreign policy (Foreign Office)

Parliamentary engagement:

  • Expert testimony to committees
  • Briefings for MPs
  • Technical translation for policymakers
  • Risk communication
  • Policy option analysis

Key Personnel and Expertise

Leadership

Ian Hogarth
Former Chair (2023-2025)
Various senior researchers
Technical leadership and research

Ian Hogarth (Former Chair, 2023-2025)

Background:

  • Technology entrepreneur and investor
  • AI safety advocate
  • Wrote influential Financial Times op-ed on "God-like AI"
  • Connections to UK tech sector and government
  • Strong safety focus

Leadership:

  • Strategic direction for AISI
  • External relations and advocacy
  • Government and international engagement
  • Representing AISI publicly
  • Ensuring independence and rigor

Approach:

  • Taking AI risks seriously
  • Building credible technical organization
  • International cooperation
  • Balancing innovation with safety
  • Evidence-based policymaking

Staffing and Expertise

Hiring from:

  • AI safety organizations (METR, Apollo, ARC, Anthropic, etc.)
  • UK universities (Oxford, Cambridge, Imperial, etc.)
  • Frontier AI labs (researchers concerned about safety)
  • Other government departments (GCHQ, etc.)
  • International talent

Expertise needed:

  • Machine learning and AI systems
  • Evaluation and red-teaming
  • Cybersecurity and CBRN
  • Interpretability and alignment
  • Policy and governance
  • International relations

Hiring challenges:

  • Competing with private sector salaries (London is expensive)
  • Government hiring processes
  • Security clearances required for some roles
  • Need for rapid scaling
  • Retention in fast-moving field

Advantages:

  • Mission-driven work with government backing
  • Influence on policy and practice
  • Access to frontier models
  • International platform
  • Job security and meaningful impact

Relationship with AI Labs

Cooperation framework:

Voluntary Agreements

Lab commitments:

  • Provide pre-release access to models
  • Share safety-relevant information
  • Participate in evaluations
  • Engage on standard-setting
  • International coordination

AISI commitments:

  • Responsible handling of sensitive information
  • Constructive engagement
  • Technical rigor
  • Timely evaluations
  • Security of proprietary data

Labs participating:

  • OpenAI
  • Anthropic
  • Google DeepMind
  • Others as they develop frontier models

Information Sharing

What labs share:

  • Model access for evaluation (pre-deployment)
  • Technical documentation
  • Safety research findings
  • Risk assessments
  • Incident reports (if any)

Confidentiality:

  • NDAs and secure handling
  • Classified information protocols
  • Public reporting (aggregated/redacted)
  • National security considerations
  • Competitive sensitivity

Mutual Benefits

For labs:

  • Independent validation of safety claims
  • Early warning of risks
  • Inform better safety practices
  • Government relationships
  • Social license and legitimacy

For AISI:

  • Access to frontier systems
  • Understanding of frontier capabilities
  • Influence on deployment decisions
  • Technical learning
  • Real-world testing

Tensions and Challenges

Potential conflicts:

  • Labs want speed, AISI wants thoroughness
  • Commercial sensitivity vs. transparency
  • National security complications
  • Regulatory uncertainty
  • International competition

Future evolution:

  • Currently voluntary, might become mandatory
  • Regulatory framework could formalize
  • International coordination might create pressure
  • Labs might resist if too burdensome
  • Balance between cooperation and oversight

International Leadership and Influence

AI Safety Summits

Bletchley Park Summit (November 2023):

  • UK convened first international AI Safety Summit
  • 28 countries and EU committed to AI safety cooperation
  • Bletchley Declaration recognized catastrophic risks
  • Foundation for international coordination
  • UK positioned as leader

Second Summit (2024):

  • South Korea and UK co-hosting
  • Building on Bletchley commitments
  • Deepening international cooperation
  • Expanding participation
  • Concrete safety measures

Ongoing convenings:

  • Regular international meetings
  • Technical working groups
  • Research collaborations
  • Standard harmonization
  • Diplomatic engagement

Setting Global Norms

UK influence:

Standard-setting:

  • Evaluation methodologies becoming international reference
  • Safety frameworks adopted by other countries
  • Best practices dissemination
  • Training and capacity building
  • Soft power through technical leadership

Bridging role:

  • Between US and Europe
  • Academic and government
  • Technical and political
  • Innovation and safety
  • National and international

Challenges:

  • China not participating fully
  • Different national approaches
  • Verification and enforcement
  • Balancing openness and security
  • Resource constraints

Impact and Accomplishments

Quantified Outcomes

AchievementImpactTimeframe
Bletchley Declaration28 countries + EU committed to AI safety cooperationNovember 2023
International Network10+ countries forming coordinated AI safety ecosystemMay 2024 onwards
Inspect AI adoptionFramework used by Anthropic, DeepMind, xAI2024-2025
Research funding28.5 million GBP+ in external grants launched2024-2025
AI Research Resource300 million GBP investment (tripled from 100M)2023-2024
Staff growth30+ technical researchers recruited2023-2024

On UK AI Policy

AISI provides technical expertise directly to Parliament and ministers, informing risk assessments for policy decisions and shaping the framework for UK AI regulation. Key policy inputs include the Online Safety Act AI provisions, AI White Paper consultation responses, regulatory framework development, and national security AI strategy.

On Industry Practices

AISI's evaluation methods are becoming industry standard. Major labs (OpenAI, Anthropic, Google DeepMind) have entered voluntary agreements for pre-deployment testing, with Inspect AI being adopted as the evaluation framework of choice. This has normalized pre-deployment safety assessment, increased transparency requirements, and established red-teaming as standard practice.

On International AI Safety

The UK is now recognized as a global leader in AI safety convening. Concrete outcomes include the Bletchley Declaration (first international agreement on AI risks), the International Network of AI Safety Institutes (10+ member countries), shared evaluation methodologies through Inspect AI, the International Scientific Report on AI Safety (led by Turing Award winner Yoshua Bengio), and foundation for ongoing governance discussions at summits in Seoul and Paris.

Criticisms and Challenges

The 2025 Rebrand Controversy

The February 2025 renaming from "AI Safety Institute" to "AI Security Institute" drew significant criticism from the AI safety community. AI Now Institute noted that the change signals the institute "will no longer focus on AI ethical issues, such as algorithm bias or protecting freedom of speech in AI applications." Critics worry this represents a narrowing of scope away from broader societal harms toward a purely national security focus.

Defenders argue the rebrand reflects AISI's actual priorities from the beginning, with Ian Hogarth emphasizing that "this change of name better reflects the team's essential remit and the priorities that they have always focused on."

Can Government Keep Pace?

AI capabilities advance faster than government institutions can respond. AISI faces structural challenges including bureaucratic constraints on hiring speed, difficulty matching private sector salaries (despite being competitive within civil service), limited compute resources compared to lab budgets, and inevitable evaluation lag behind the frontier.

AISI's response has been to operate as a "startup inside government" with fast-track hiring, partnerships with academic and private sector organizations, and focus on high-impact evaluation work rather than trying to match lab resources directly.

Regulatory Capture Risk

There are concerns that AISI might be captured by industry interests given its dependence on voluntary lab cooperation, the information asymmetry favoring labs, and the revolving door between government and industry. However, mitigations include government independence, diverse expertise sourcing (not only from labs), public interest mandate, parliamentary oversight, and increasing transparency.

International Coordination Challenges

China's absence from Western AI safety coordination remains a significant limitation. The International Network of AI Safety Institutes includes primarily democratic allies, with no clear pathway to engaging China despite early participation in the Bletchley Declaration. This creates risks of fragmented governance and potential race dynamics.

Resources and Scope

With an annual budget of approximately 50 million GBP, AISI remains smaller than major lab research budgets and must prioritize carefully. The Institute cannot evaluate everything and depends heavily on partnerships. Maintaining quality and credibility while scaling remains an ongoing challenge.

Key Questions

  • ?Can UK maintain global leadership in AI safety long-term?
  • ?Will voluntary lab cooperation be sufficient or is regulation needed?
  • ?How to coordinate internationally without China participation?
  • ?Can government evaluation keep pace with frontier AI development?
  • ?What enforcement mechanisms should back UK AI safety standards?
  • ?How to balance UK national interests with international cooperation?

Future Directions

Near-Term (1-2 years)

Organizational development:

  • Complete initial hiring and team building
  • Establish robust evaluation infrastructure
  • Deepen lab partnerships
  • Expand international collaboration
  • Demonstrate technical credibility

Key deliverables:

  • Regular frontier model evaluations
  • Published research and standards
  • Second AI Safety Summit outcomes
  • International coordination agreements
  • Policy recommendations to UK government

Medium-Term (2-5 years)

Institutionalization:

  • Mature organization with stable funding
  • Recognized global leader in AI safety
  • Influential in international standards
  • Comprehensive evaluation program
  • Growing team and capabilities

Possible developments:

  • Regulatory authority (if legislation passed)
  • Mandatory evaluation requirements
  • International verification mechanisms
  • Expanded scope (multimodal, robotics, etc.)
  • Regional presence beyond London

Long-Term Vision

UK as global AI safety hub:

  • Leading technical expertise
  • Convening power for international cooperation
  • Standards and norms shaping global governance
  • Preventing catastrophic AI deployments
  • Contributing to beneficial AI development

Broader impact:

  • Effective international AI governance
  • Safe development of transformative AI
  • UK playing key role in existential risk reduction
  • Model for other countries
  • Public trust in AI governance

Perspectives on UK AISI

UK AISI's Role and Effectiveness

Essential Global Leadership

UK AISI is crucial for international AI safety coordination. Government leadership necessary. Evaluation and standard-setting valuable. Should have regulatory authority. Model for other countries.

Proponents: Many safety researchers, UK government, International cooperation advocates
Confidence: high (4/5)
Valuable but Limited

AISI useful for research and coordination but limited by resources and authority. Can't compete with labs technically. Voluntary cooperation might not be sufficient. Need stronger enforcement.

Proponents: Some safety researchers, Cautious observers
Confidence: medium (3/5)
Risk of Capture

Government institute will be captured by industry. Labs have information and resource advantage. AISI will rubber-stamp industry preferences. Need truly independent oversight.

Proponents: Skeptics of government-industry partnerships
Confidence: low (2/5)
Innovation Concerns

Government involvement might slow beneficial AI development. Bureaucracy and risk-aversion problematic. Industry self-regulation preferable. AISI could overreach and harm UK AI sector.

Proponents: Some industry voices, Innovation-focused advocates
Confidence: low (1/5)

Comparisons to Other Organizations

vs US AI Safety Institute

Similarities:

  • Both government AI safety bodies
  • Similar missions and approaches
  • Close coordination and partnership
  • Created around same time (2023)

Differences:

  • UK: Smaller budget and country, more nimble
  • US: Larger resources, broader scope
  • UK: Explicit international hub strategy
  • US: Within NIST, different structure
  • UK: More unified government approach
  • US: More complex federal system

Relationship: Sister organizations, very close collaboration

vs Private Safety Organizations

Complementary:

  • METR, Apollo, ARC: Private orgs focused on research and evaluation
  • UK AISI: Government body with policy authority
  • AISI learns from and hires from safety orgs
  • Different authorities and mandates
  • Collaborative ecosystem

vs Frontier Labs

Different roles:

  • Labs: Build AI systems, commercial
  • AISI: Evaluate and oversee, government
  • Labs: Innovation focus
  • AISI: Safety focus
  • Cooperative but some tension
  • AISI provides independent assessment

Key Developments Timeline

DateEvent
June 2023Frontier AI Taskforce launched; Ian Hogarth appointed as Chair
November 2023AI Safety Summit at Bletchley Park; 28 countries sign Bletchley Declaration
November 2023UK announces creation of AI Safety Institute
April 2024First evaluation results published on publicly available frontier models
May 2024Inspect AI evaluation framework open-sourced
May 2024San Francisco office announced
May 2024AI Seoul Summit: International Network of AI Safety Institutes formed
May 2024International Scientific Report on AI Safety published (led by Yoshua Bengio)
November 2024First meeting of International Network in San Francisco (10 countries + EU)
December 2024Frontier AI Trends Report published
February 2025Renamed to AI Security Institute at Munich Security Conference
February 2025Criminal misuse team launched with Home Office

Sources and Resources

Official Sources

  • UK AI Security Institute Official Website - Main portal for the Institute
  • GOV.UK AI Safety Institute Page - Government organizational page
  • Introducing the AI Safety Institute - Official overview document
  • Fourth Progress Report - May 2024 progress update
  • Our 2025 Year in Review - 2025 annual review

Research Outputs

  • Frontier AI Trends Report - December 2024 assessment of frontier AI capabilities
  • Inspect AI Framework - Open-source evaluation framework documentation
  • Inspect Evals - Collection of 100+ pre-built evaluations
  • Early Lessons from Evaluating Frontier AI Systems - Evaluation methodology insights

Funding Programs

  • The Alignment Project - 15 million GBP global alignment research fund
  • Grants Overview - Information on all AISI grant programs
  • Systemic Safety Grants - 8.5 million GBP societal resilience program

International Coordination

  • The Bletchley Declaration - November 2023 international agreement
  • Chair's Summary of AI Safety Summit 2023 - Summit outcomes
  • CSIS: The AI Safety Institute International Network - Analysis of international coordination
  • Centre for Future Generations: The AI Safety Institute Network - Comparative analysis

News and Analysis

  • UK AI Safety Institute Wikipedia - Comprehensive background
  • Computer Weekly: San Francisco Office Announcement - May 2024 expansion
  • Infosecurity Magazine: AISI Rebrands - February 2025 name change
  • AI Now Statement on Transition - Critical perspective on rebrand

References

This CSIS analysis examines the international network of AI Safety Institutes established across multiple countries and provides recommendations for strengthening their coordination, scope, and effectiveness. It addresses how these institutes can better collaborate on technical safety evaluations and policy alignment to address frontier AI risks.

★★★★☆
2UK AI Security Institute's evaluationsUK AI Safety Institute·Government

The UK AI Safety Institute shares early findings and methodology from its evaluations of frontier AI models, covering how they assess potentially dangerous capabilities including cybersecurity risks, CBRN threats, and autonomous behavior. The post outlines the AISI's approach to pre-deployment evaluations and the practical challenges encountered when testing leading AI systems.

★★★★☆
3government AI policiesUK Government·Government

The Bletchley Declaration is a landmark multinational policy agreement signed at the AI Safety Summit 2023, committing participating nations to collaborative efforts on AI safety while enabling beneficial AI development. It represents one of the first major intergovernmental consensus documents explicitly addressing risks from frontier AI systems, including potential catastrophic and existential harms.

★★★★☆
4The Alignment ProjectUK Government·Government

The Alignment Project is a major international funding initiative launched in 2025 by the UK AI Security Institute and a broad coalition of government, industry, and philanthropic partners to accelerate AI alignment research. In its first round, it awarded over £27 million to 60+ projects, with grants up to £1 million supplemented by compute access, expert support, and potential venture capital. The initiative aims to ensure advanced AI systems remain safe, reliable, and under human control.

★★★★☆
5Our 2025 Year in ReviewUK AI Safety Institute·Government

The UK AI Security Institute (AISI) reviews its 2025 achievements, including publishing the first Frontier AI Trends Report based on two years of testing over 30 frontier AI systems. Key advances include deepened evaluation suites across cyber, chem-bio, and alignment domains, plus pioneering work on sandbagging detection, self-replication benchmarks, and AI-enabled persuasion research published in Science.

★★★★☆

Inspect Evals is a repository of community-contributed AI model evaluations maintained by the UK AI Safety Institute (AISI), featuring implementations of popular benchmarks across coding, reasoning, safety, and agent capabilities. Evaluations can be installed and run with a single command against any model, covering domains from Python coding to cybersecurity to scientific reasoning.

7Introducing the UK AI Safety InstituteUK Government·Government

The UK government's foundational document introducing the AI Safety Institute (AISI), the first state-backed organization dedicated to advanced AI safety for the public interest. It outlines AISI's mission to minimize surprise from rapid AI advances and develop sociotechnical infrastructure to understand and mitigate AI risks, presented to Parliament in November 2023 by Ian Hogarth.

★★★★☆
8Systemic Safety GrantsUK AI Safety Institute·Government

The UK AI Safety Institute launched a Systemic AI Safety Grants programme offering up to £200,000 to researchers from academia, industry, and civil society. The programme targets broader societal risks from AI deployment—including sector-specific impacts in healthcare, education, and finance, multi-agent interactions, and infrastructure vulnerabilities—beyond individual model capabilities. It aims to map research priorities, develop new methodologies, and inform government policy ahead of anticipated rapid AI adoption over the next 2-5 years.

★★★★☆

This is the official UK government Chair's statement from the inaugural AI Safety Summit held at Bletchley Park on 1-2 November 2023. The summit brought together international stakeholders to identify next steps for the safe development of frontier AI. It represents a landmark moment in international AI governance coordination, resulting in the Bletchley Declaration.

★★★★☆
10AISI Frontier AI TrendsUK AI Safety Institute·Government

A UK AI Safety Institute government assessment documenting exponential performance improvements across frontier AI systems in multiple domains. The report evaluates emerging capabilities and associated risks, calling for robust safeguards as systems advance rapidly. It serves as an official benchmark of the current frontier AI landscape from a national safety authority.

★★★★☆
11Infosecurity Magazine: AISI Rebrandsinfosecurity-magazine.com

The UK's AI Safety Institute (AISI) underwent a rebranding as part of a broader government strategy shift in AI policy, reflecting changes in the UK's approach to AI governance and safety. The article covers the institutional reorganization and its implications for the UK's AI safety landscape.

12AI Safety Institute - GOV.UKUK Government·Government

The UK AI Safety Institute (recently rebranded as the AI Security Institute) is a government body under the Department for Science, Innovation and Technology focused on minimizing risks from rapid and unexpected AI advances. It conducts and publishes safety research, international coordination reports, and policy guidance, while managing grants for systemic AI safety research.

★★★★☆
13UK AI Safety Institute WikipediaWikipedia·Reference

Wikipedia article covering the UK AI Safety Institute (AISI), a government body established in 2023 to advance AI safety research and evaluation. It provides an overview of the institute's mission, structure, key activities such as frontier model evaluations, and its role in international AI safety coordination. The article serves as a reference point for understanding the UK's institutional approach to governing advanced AI.

★★★☆☆
14AISI Grants Programs OverviewUK AI Safety Institute·Government

The UK AI Safety Institute (AISI) has administered over £15 million in grants across three programs to advance AI safety and security research. Programs include the Alignment Project (up to £1M per project), Challenge Fund (up to £200K per project), and Systemic AI Safety Grants focused on societal resilience. All programs are currently closed to new applications.

★★★★☆

The AI Now Institute responds to the UK government's decision to rename and reorient the AI Safety Institute into the AI Security Institute, expressing concern that this shift deprioritizes broader societal safety in favor of narrower security framing. The statement argues this transition risks sidelining critical work on AI harms affecting workers, marginalized communities, and democratic institutions. It calls for maintaining robust, independent AI safety oversight.

★★★★☆

The UK AI Safety Institute (AISI) announced plans to open a branch office in San Francisco, expanding its international presence into a major hub of AI development. This move signals the UK government's intent to maintain close engagement with leading AI companies and researchers based in Silicon Valley, strengthening its role in global AI safety governance.

17Fourth Progress ReportUK AI Safety Institute·Government

The UK AI Safety Institute's fourth progress report documents its rapid institutional growth and key achievements through May 2024, including coordinating the first International Scientific Report on Advanced AI Safety across 30 countries, open-sourcing the Inspect evaluation framework, and expanding to over 30 researchers. The report illustrates how a government body can scale from a single researcher to a significant AI safety evaluation organization within a year.

★★★★☆

This Centre for Future Generations resource analyzes the emerging international network of AI Safety Institutes, examining their membership, mandates, and coordination mechanisms. It provides an overview of how national AI safety bodies are structured and how they collaborate across borders to address shared risks from advanced AI systems.

Inspect is an open-source framework developed by the UK AI Safety Institute (AISI) for evaluating large language models and AI systems. It provides standardized tools for running safety evaluations, benchmarks, and red-teaming tasks. The framework enables researchers and developers to assess AI model capabilities and safety properties in a reproducible and extensible way.

20UK AI Safety Institute (AISI)UK AI Safety Institute·Government

The UK AI Safety Institute (AISI) is the UK government's dedicated body for evaluating and mitigating risks from advanced AI systems. It conducts technical safety research, develops evaluation frameworks for frontier AI models, and works with international partners to inform global AI governance and policy.

★★★★☆

Structured Data

1 fact·1 recordView in FactBase →

Key People

1
IH
Ian Hogarth
Chair · Jun 2023–Feb 2025

All Facts

1
General
PropertyValueAs OfSource
Websitehttps://www.aisi.gov.uk

Related Wiki Pages

Top Related Pages

Approaches

AI EvaluationAI Governance Coordination Technologies

Analysis

Intervention Timing WindowsAI Safety Research Allocation Model

Policy

Bletchley DeclarationAI Safety Institutes (AISIs)

Organizations

Anthropic

Concepts

International Coordination MechanismsCompute ThresholdsAutonomous CodingSituational Awareness

Other

AI EvaluationsIan HogarthRed TeamingOliver Sourbut

Historical

AI Safety Summit (Bletchley Park)

Risks

Bioweapons RiskAI Knowledge Monopoly

Key Debates

AI Accident Risk CruxesAI Governance and Policy