UK AI Safety Institute
UK AI Safety Institute
The UK AI Safety Institute (renamed AI Security Institute in Feb 2025) operates with ~30 technical staff and 50M GBP annual budget, conducting frontier model evaluations using its open-source Inspect AI framework and coordinating the 10+ country International Network of AI Safety Institutes. April 2024 evaluations found frontier models capable of intermediate cybersecurity tasks and PhD-level biology knowledge, with safeguards vulnerable to basic jailbreaks.
Overview
The UK AI Safety Institute (UK AISI) is a government organization established in November 2023 to advance AI safety through research, evaluation, and international coordination. Created in the wake of the first AI Safety Summit at Bletchley Park, AISI represents the UK's commitment to being a global leader in AI safety and governance. In February 2025, the organization was renamed the AI Security Institute to reflect a sharper focus on national security threats including cyberattacks, fraud, and AI-enabled bioweapon development.
As one of the first government bodies specifically dedicated to advanced AI safety (alongside the US AISI), the UK AISI plays a pioneering role in translating AI safety research into government policy and practice. The organization conducts technical research on AI risks, evaluates frontier models for dangerous capabilities, develops safety standards, and coordinates international efforts through the International Network of AI Safety Institutes, which now includes 10+ member countries.
The UK AISI's strategic positioning reflects the UK government's ambition: to be a global hub for AI safety, bridging technical research communities, frontier AI labs, and international policymakers. Former Chair Ian Hogarth described AISI as a "startup inside the government" that combines technical rigor with policy influence.
Organization at a Glance
| Attribute | Details |
|---|---|
| Founded | November 2023 (following AI Safety Summit) |
| Renamed | February 2025 (to AI Security Institute) |
| Parent Department | Department for Science, Innovation and Technology (DSIT) |
| Leadership | Ian Hogarth (former Chair, 2023-2025) |
| Technical Staff | 30+ researchers (as of mid-2024) |
| Annual Budget | Approximately 50 million GBP |
| Locations | London (HQ), San Francisco (opened 2024) |
| Key Output | Inspect AI framework, Frontier AI Trends Report |
| International Role | Founding member of International Network of AI Safety Institutes |
History and Founding
Background: UK AI Safety Summit (November 2023)
Bletchley Park Declaration:
- UK hosted first international AI Safety Summit at Bletchley Park
- 28 countries and EU signed declaration on AI risks
- Recognized catastrophic risks from advanced AI
- Committed to international cooperation on safety
- Political moment crystallizing government attention
Summit outcomes:
- Agreement on establishing AI Safety Institutes
- UK and US announced creation of their institutes
- Commitments from frontier AI labs
- International research collaboration agreements
- Foundation for ongoing coordination
Context:
- ChatGPT moment (late 2022) raised AI awareness
- Growing concern about AI existential risk
- UK positioning for leadership role
- Rishi Sunak government prioritizing AI policy
- Academic and researcher advocacy for government action
Establishment (2023)
Founding: Announced at AI Safety Summit, operationalized late 2023
Organizational placement:
- Part of UK Department for Science, Innovation and Technology (DSIT)
- Government agency with civil service staff
- Independent research capacity
- Policy coordination role
- International engagement mandate
Initial mission:
- Conduct safety research on advanced AI
- Evaluate frontier models for dangerous capabilities
- Develop testing and evaluation methodologies
- Coordinate international AI safety efforts
- Advise UK government on AI risks and governance
Funding: The UK initially invested 100 million GBP in the Frontier AI Taskforce (AISI's predecessor), providing more funding for AI safety than any other country at the time. The Institute's annual budget is approximately 50 million GBP, continuing as an annual amount through the end of the decade, subject to demonstrating continued requirement. This makes the UK AISI an outlier in funding compared to other national AI Safety Institutes globally.
Early Development (2023-2024)
2023 activities: The Institute rapidly hired technical staff, recruiting senior alumni from OpenAI, Google DeepMind, and the University of Oxford. Ian Hogarth was appointed as Chair in June 2023 and served until 2025. By the end of 2023, the organization had established its organizational structure, built relationships with AI labs for model access, and begun coordination with the US AISI.
2024 milestones:
| Date | Development |
|---|---|
| April 2024 | Published first evaluation results on publicly available frontier models |
| May 2024 | Open-sourced Inspect AI evaluation framework |
| May 2024 | Announced San Francisco office opening |
| May 2024 | AI Seoul Summit: formed International Network of AI Safety Institutes |
| November 2024 | First meeting of International Network in San Francisco |
| December 2024 | Published Frontier AI Trends Report |
Current status (2025): Following the February 2025 renaming to AI Security Institute, the organization now focuses more explicitly on national security threats. Technology Secretary Peter Kyle announced the change at the Munich Security Conference, emphasizing a pivot from concerns about bias and misinformation toward risks such as cyberattacks, fraud, and AI-enabled bioweapon development. The Institute has launched a new criminal misuse team in partnership with the Home Office to investigate AI's role in serious crimes including child exploitation, cybercrime, and financial fraud.
Core Functions and Activities
1. Frontier Model Evaluations
The Institute independently assesses advanced AI systems for dangerous capabilities across domains critical to national security and public safety. In their April 2024 evaluation of publicly available frontier models, AISI found several concerning capabilities:
| Evaluation Domain | Key Findings (April 2024) |
|---|---|
| Cybersecurity | Models capable of completing basic to intermediate cybersecurity challenges |
| Chemistry/Biology | Several models demonstrated PhD-equivalent level of knowledge |
| Safeguards | Built-in safeguards vulnerable to even basic "jailbreaks" |
| Autonomous Systems | Models performed better than biology PhD experts on open-ended questions |
Methodology: The Institute has built evaluation expertise by learning from and collaborating with organizations like METR, Apollo Research, and ARC. Their approach includes both pre-deployment and post-deployment testing, with a focus on red-teaming and adversarial evaluation techniques.
Lab cooperation: AISI has established voluntary agreements with OpenAI, Anthropic, and Google DeepMind for pre-release access to models. Labs share safety-relevant information under NDAs, coordinate on safety thresholds, and collaborate on evaluation methodology development.
Inspect AI Framework: In May 2024, AISI open-sourced Inspect AI, a Python framework for building and running reproducible LLM evaluations. The framework has been adopted by major AI labs including Anthropic, DeepMind, and xAI. Inspect AI includes over 100 pre-built evaluations, a web-based visualization tool, and support for tool calling and multi-agent evaluations. Key benchmarks include ChemBench (2,786 chemistry questions), Humanity's Last Exam (3,000 questions across dozens of subjects), and Cybench (40 professional-level cybersecurity challenges).
2. Research Funding Programs
AISI manages substantial external research funding, making it one of the largest funders of AI safety research globally. The Institute's grant programs combine government funding with partnerships from industry and philanthropy.
| Program | Total Funding | Per-Project Range | Focus Areas |
|---|---|---|---|
| The Alignment Project | 15 million GBP | 50K-1M GBP | Alignment research, interdisciplinary collaboration |
| Systemic Safety Grants | 8.5 million GBP | Up to 200K GBP | Societal resilience, healthcare, energy grids, financial markets |
| Challenge Fund | 5 million GBP | Up to 200K GBP | Safeguards, control, alignment, societal resilience |
The Alignment Project: This global fund is supported by an international coalition including the UK AISI, Canadian AI Safety Institute, Schmidt Sciences, Amazon Web Services, Halcyon Futures, Safe AI Fund, UKRI, Anthropic, and ARIA. Recipients can also receive up to 5 million GBP in dedicated AWS cloud computing credits, enabling technical experiments beyond typical academic reach.
Systemic Safety Grants: Launched in partnership with the Engineering and Physical Sciences Research Council (EPSRC) and Innovate UK (part of UKRI), the first phase allocated 4 million GBP to support approximately 20 projects. The program aims to increase societal resilience to widespread AI deployment across critical infrastructure.
3. Technical Safety Research
AISI conducts both in-house and collaborative research across several priority areas:
Technical research: Evaluation methodologies, interpretability and transparency techniques, robustness and reliability testing, alignment approaches, and measurement/benchmarking standards.
Sociotechnical research: AI governance frameworks, risk assessment methodologies, safety standards and best practices, international coordination mechanisms, and public engagement strategies.
Key outputs include: The Frontier AI Trends Report (December 2024), which provides an empirical assessment of frontier AI capabilities and their trajectory, and regular evaluation reports shared with policymakers and the public.
4. International Coordination
The UK has positioned itself as a convener for international AI safety efforts, hosting the first AI Safety Summit and co-founding the International Network of AI Safety Institutes.
Diagram (loading…)
flowchart TD UK[UK AI Security Institute] --> NETWORK[International Network of<br/>AI Safety Institutes] US[US AI Safety Institute] --> NETWORK NETWORK --> EU[European Commission] NETWORK --> JP[Japan] NETWORK --> KR[Republic of Korea] NETWORK --> SG[Singapore] NETWORK --> CA[Canada] NETWORK --> AU[Australia] NETWORK --> FR[France] NETWORK --> KE[Kenya] UK --> SF[San Francisco Office] SF -.->|Close collaboration| US style UK fill:#1e40af,color:#fff style NETWORK fill:#059669,color:#fff
International Network of AI Safety Institutes: During the AI Seoul Summit in May 2024, international leaders agreed to form this network comprising institutes from the UK, US, Japan, France, Germany, Italy, Singapore, South Korea, Australia, Canada, and the European Union. The first meeting was held in San Francisco in November 2024, with representatives from 10 countries and the European Commission.
US-UK partnership: The UK and US AISIs work as "sister organizations" with close collaboration on methodologies, coordinated evaluations, and joint international engagement. The San Francisco office (opened summer 2024) facilitates this partnership and allows AISI to recruit Bay Area talent and engage directly with major AI labs.
Seoul Statement goals: The international network aims to "accelerate the advancement of the science of AI safety" by promoting complementarity and interoperability between institutes and fostering a common international understanding of AI safety approaches.
5. Standard-Setting and Guidelines
Developing best practices:
Evaluation standards:
- Methodologies for dangerous capability testing
- Thresholds for deployment decisions
- Red-teaming protocols
- Transparency and documentation requirements
- Continuous monitoring approaches
Safety standards:
- Risk management frameworks
- Safety assurance processes
- Incident response and reporting
- Governance and accountability
- Alignment verification
UK and international adoption:
- UK government procurement requirements
- Industry adoption (voluntary initially)
- International harmonization efforts
- Foundation for potential regulation
- Soft power and norm-setting
6. Policy Advice and Government Coordination
Advising UK government:
Policy development:
- Risk assessments for ministers and parliament
- Technical input to AI legislation
- Regulatory framework design
- International negotiation support
- Public communication strategies
Interagency coordination:
- National security agencies (GCHQ, MI5, MI6)
- Regulatory bodies (ICO, CMA, Ofcom)
- Research funders (UKRI)
- Science and technology departments
- Foreign policy (Foreign Office)
Parliamentary engagement:
- Expert testimony to committees
- Briefings for MPs
- Technical translation for policymakers
- Risk communication
- Policy option analysis
Key Personnel and Expertise
Leadership
Ian Hogarth (Former Chair, 2023-2025)
Background:
- Technology entrepreneur and investor
- AI safety advocate
- Wrote influential Financial Times op-ed on "God-like AI"
- Connections to UK tech sector and government
- Strong safety focus
Leadership:
- Strategic direction for AISI
- External relations and advocacy
- Government and international engagement
- Representing AISI publicly
- Ensuring independence and rigor
Approach:
- Taking AI risks seriously
- Building credible technical organization
- International cooperation
- Balancing innovation with safety
- Evidence-based policymaking
Staffing and Expertise
Hiring from:
- AI safety organizations (METR, Apollo, ARC, Anthropic, etc.)
- UK universities (Oxford, Cambridge, Imperial, etc.)
- Frontier AI labs (researchers concerned about safety)
- Other government departments (GCHQ, etc.)
- International talent
Expertise needed:
- Machine learning and AI systems
- Evaluation and red-teaming
- Cybersecurity and CBRN
- Interpretability and alignment
- Policy and governance
- International relations
Hiring challenges:
- Competing with private sector salaries (London is expensive)
- Government hiring processes
- Security clearances required for some roles
- Need for rapid scaling
- Retention in fast-moving field
Advantages:
- Mission-driven work with government backing
- Influence on policy and practice
- Access to frontier models
- International platform
- Job security and meaningful impact
Relationship with AI Labs
Cooperation framework:
Voluntary Agreements
Lab commitments:
- Provide pre-release access to models
- Share safety-relevant information
- Participate in evaluations
- Engage on standard-setting
- International coordination
AISI commitments:
- Responsible handling of sensitive information
- Constructive engagement
- Technical rigor
- Timely evaluations
- Security of proprietary data
Labs participating:
- OpenAI
- Anthropic
- Google DeepMind
- Others as they develop frontier models
Information Sharing
What labs share:
- Model access for evaluation (pre-deployment)
- Technical documentation
- Safety research findings
- Risk assessments
- Incident reports (if any)
Confidentiality:
- NDAs and secure handling
- Classified information protocols
- Public reporting (aggregated/redacted)
- National security considerations
- Competitive sensitivity
Mutual Benefits
For labs:
- Independent validation of safety claims
- Early warning of risks
- Inform better safety practices
- Government relationships
- Social license and legitimacy
For AISI:
- Access to frontier systems
- Understanding of frontier capabilities
- Influence on deployment decisions
- Technical learning
- Real-world testing
Tensions and Challenges
Potential conflicts:
- Labs want speed, AISI wants thoroughness
- Commercial sensitivity vs. transparency
- National security complications
- Regulatory uncertainty
- International competition
Future evolution:
- Currently voluntary, might become mandatory
- Regulatory framework could formalize
- International coordination might create pressure
- Labs might resist if too burdensome
- Balance between cooperation and oversight
International Leadership and Influence
AI Safety Summits
Bletchley Park Summit (November 2023):
- UK convened first international AI Safety Summit
- 28 countries and EU committed to AI safety cooperation
- Bletchley Declaration recognized catastrophic risks
- Foundation for international coordination
- UK positioned as leader
Second Summit (2024):
- South Korea and UK co-hosting
- Building on Bletchley commitments
- Deepening international cooperation
- Expanding participation
- Concrete safety measures
Ongoing convenings:
- Regular international meetings
- Technical working groups
- Research collaborations
- Standard harmonization
- Diplomatic engagement
Setting Global Norms
UK influence:
Standard-setting:
- Evaluation methodologies becoming international reference
- Safety frameworks adopted by other countries
- Best practices dissemination
- Training and capacity building
- Soft power through technical leadership
Bridging role:
- Between US and Europe
- Academic and government
- Technical and political
- Innovation and safety
- National and international
Challenges:
- China not participating fully
- Different national approaches
- Verification and enforcement
- Balancing openness and security
- Resource constraints
Impact and Accomplishments
Quantified Outcomes
| Achievement | Impact | Timeframe |
|---|---|---|
| Bletchley Declaration | 28 countries + EU committed to AI safety cooperation | November 2023 |
| International Network | 10+ countries forming coordinated AI safety ecosystem | May 2024 onwards |
| Inspect AI adoption | Framework used by Anthropic, DeepMind, xAI | 2024-2025 |
| Research funding | 28.5 million GBP+ in external grants launched | 2024-2025 |
| AI Research Resource | 300 million GBP investment (tripled from 100M) | 2023-2024 |
| Staff growth | 30+ technical researchers recruited | 2023-2024 |
On UK AI Policy
AISI provides technical expertise directly to Parliament and ministers, informing risk assessments for policy decisions and shaping the framework for UK AI regulation. Key policy inputs include the Online Safety Act AI provisions, AI White Paper consultation responses, regulatory framework development, and national security AI strategy.
On Industry Practices
AISI's evaluation methods are becoming industry standard. Major labs (OpenAI, Anthropic, Google DeepMind) have entered voluntary agreements for pre-deployment testing, with Inspect AI being adopted as the evaluation framework of choice. This has normalized pre-deployment safety assessment, increased transparency requirements, and established red-teaming as standard practice.
On International AI Safety
The UK is now recognized as a global leader in AI safety convening. Concrete outcomes include the Bletchley Declaration (first international agreement on AI risks), the International Network of AI Safety Institutes (10+ member countries), shared evaluation methodologies through Inspect AI, the International Scientific Report on AI Safety (led by Turing Award winner Yoshua Bengio), and foundation for ongoing governance discussions at summits in Seoul and Paris.
Criticisms and Challenges
The 2025 Rebrand Controversy
The February 2025 renaming from "AI Safety Institute" to "AI Security Institute" drew significant criticism from the AI safety community. AI Now Institute noted that the change signals the institute "will no longer focus on AI ethical issues, such as algorithm bias or protecting freedom of speech in AI applications." Critics worry this represents a narrowing of scope away from broader societal harms toward a purely national security focus.
Defenders argue the rebrand reflects AISI's actual priorities from the beginning, with Ian Hogarth emphasizing that "this change of name better reflects the team's essential remit and the priorities that they have always focused on."
Can Government Keep Pace?
AI capabilities advance faster than government institutions can respond. AISI faces structural challenges including bureaucratic constraints on hiring speed, difficulty matching private sector salaries (despite being competitive within civil service), limited compute resources compared to lab budgets, and inevitable evaluation lag behind the frontier.
AISI's response has been to operate as a "startup inside government" with fast-track hiring, partnerships with academic and private sector organizations, and focus on high-impact evaluation work rather than trying to match lab resources directly.
Regulatory Capture Risk
There are concerns that AISI might be captured by industry interests given its dependence on voluntary lab cooperation, the information asymmetry favoring labs, and the revolving door between government and industry. However, mitigations include government independence, diverse expertise sourcing (not only from labs), public interest mandate, parliamentary oversight, and increasing transparency.
International Coordination Challenges
China's absence from Western AI safety coordination remains a significant limitation. The International Network of AI Safety Institutes includes primarily democratic allies, with no clear pathway to engaging China despite early participation in the Bletchley Declaration. This creates risks of fragmented governance and potential race dynamics.
Resources and Scope
With an annual budget of approximately 50 million GBP, AISI remains smaller than major lab research budgets and must prioritize carefully. The Institute cannot evaluate everything and depends heavily on partnerships. Maintaining quality and credibility while scaling remains an ongoing challenge.
Key Questions
- ?Can UK maintain global leadership in AI safety long-term?
- ?Will voluntary lab cooperation be sufficient or is regulation needed?
- ?How to coordinate internationally without China participation?
- ?Can government evaluation keep pace with frontier AI development?
- ?What enforcement mechanisms should back UK AI safety standards?
- ?How to balance UK national interests with international cooperation?
Future Directions
Near-Term (1-2 years)
Organizational development:
- Complete initial hiring and team building
- Establish robust evaluation infrastructure
- Deepen lab partnerships
- Expand international collaboration
- Demonstrate technical credibility
Key deliverables:
- Regular frontier model evaluations
- Published research and standards
- Second AI Safety Summit outcomes
- International coordination agreements
- Policy recommendations to UK government
Medium-Term (2-5 years)
Institutionalization:
- Mature organization with stable funding
- Recognized global leader in AI safety
- Influential in international standards
- Comprehensive evaluation program
- Growing team and capabilities
Possible developments:
- Regulatory authority (if legislation passed)
- Mandatory evaluation requirements
- International verification mechanisms
- Expanded scope (multimodal, robotics, etc.)
- Regional presence beyond London
Long-Term Vision
UK as global AI safety hub:
- Leading technical expertise
- Convening power for international cooperation
- Standards and norms shaping global governance
- Preventing catastrophic AI deployments
- Contributing to beneficial AI development
Broader impact:
- Effective international AI governance
- Safe development of transformative AI
- UK playing key role in existential risk reduction
- Model for other countries
- Public trust in AI governance
Perspectives on UK AISI
UK AISI's Role and Effectiveness
UK AISI is crucial for international AI safety coordination. Government leadership necessary. Evaluation and standard-setting valuable. Should have regulatory authority. Model for other countries.
AISI useful for research and coordination but limited by resources and authority. Can't compete with labs technically. Voluntary cooperation might not be sufficient. Need stronger enforcement.
Government institute will be captured by industry. Labs have information and resource advantage. AISI will rubber-stamp industry preferences. Need truly independent oversight.
Government involvement might slow beneficial AI development. Bureaucracy and risk-aversion problematic. Industry self-regulation preferable. AISI could overreach and harm UK AI sector.
Comparisons to Other Organizations
vs US AI Safety Institute
Similarities:
- Both government AI safety bodies
- Similar missions and approaches
- Close coordination and partnership
- Created around same time (2023)
Differences:
- UK: Smaller budget and country, more nimble
- US: Larger resources, broader scope
- UK: Explicit international hub strategy
- US: Within NIST, different structure
- UK: More unified government approach
- US: More complex federal system
Relationship: Sister organizations, very close collaboration
vs Private Safety Organizations
Complementary:
- METR, Apollo, ARC: Private orgs focused on research and evaluation
- UK AISI: Government body with policy authority
- AISI learns from and hires from safety orgs
- Different authorities and mandates
- Collaborative ecosystem
vs Frontier Labs
Different roles:
- Labs: Build AI systems, commercial
- AISI: Evaluate and oversee, government
- Labs: Innovation focus
- AISI: Safety focus
- Cooperative but some tension
- AISI provides independent assessment
Key Developments Timeline
| Date | Event |
|---|---|
| June 2023 | Frontier AI Taskforce launched; Ian Hogarth appointed as Chair |
| November 2023 | AI Safety Summit at Bletchley Park; 28 countries sign Bletchley Declaration |
| November 2023 | UK announces creation of AI Safety Institute |
| April 2024 | First evaluation results published on publicly available frontier models |
| May 2024 | Inspect AI evaluation framework open-sourced |
| May 2024 | San Francisco office announced |
| May 2024 | AI Seoul Summit: International Network of AI Safety Institutes formed |
| May 2024 | International Scientific Report on AI Safety published (led by Yoshua Bengio) |
| November 2024 | First meeting of International Network in San Francisco (10 countries + EU) |
| December 2024 | Frontier AI Trends Report published |
| February 2025 | Renamed to AI Security Institute at Munich Security Conference |
| February 2025 | Criminal misuse team launched with Home Office |
Sources and Resources
Official Sources
- UK AI Security Institute Official Website↗🏛️ government★★★★☆UK AI Safety InstituteUK AI Safety Institute (AISI)AISI is a key institutional actor in AI safety, representing one of the first government-led efforts to systematically evaluate frontier AI models; its work and publications are directly relevant to governance, evaluation methodology, and international AI safety coordination.The UK AI Safety Institute (AISI) is the UK government's dedicated body for evaluating and mitigating risks from advanced AI systems. It conducts technical safety research, deve...ai-safetygovernancepolicyevaluation+5Source ↗ - Main portal for the Institute
- GOV.UK AI Safety Institute Page↗🏛️ government★★★★☆UK GovernmentAI Safety Institute - GOV.UKThis is the official UK government hub for AI safety policy and research; important for tracking state-level institutional responses to frontier AI risks and international safety coordination efforts.The UK AI Safety Institute (recently rebranded as the AI Security Institute) is a government body under the Department for Science, Innovation and Technology focused on minimizi...ai-safetygovernancepolicyevaluation+4Source ↗ - Government organizational page
- Introducing the AI Safety Institute↗🏛️ government★★★★☆UK GovernmentIntroducing the UK AI Safety InstituteThe founding document of the UK AI Safety Institute, a landmark in state-level AI governance, establishing institutional framing and mission priorities that influenced subsequent international AI safety efforts including the Bletchley Declaration.The UK government's foundational document introducing the AI Safety Institute (AISI), the first state-backed organization dedicated to advanced AI safety for the public interest...governancepolicyai-safetyevaluation+4Source ↗ - Official overview document
- Fourth Progress Report↗🏛️ government★★★★☆UK AI Safety InstituteFourth Progress ReportOfficial progress report from the UK AISI (now AI Security Institute), useful for tracking the development of government AI safety infrastructure and international coordination efforts following the 2023 Bletchley Park AI Safety Summit.The UK AI Safety Institute's fourth progress report documents its rapid institutional growth and key achievements through May 2024, including coordinating the first Internationa...ai-safetygovernanceevaluationpolicy+4Source ↗ - May 2024 progress update
- Our 2025 Year in Review↗🏛️ government★★★★☆UK AI Safety InstituteOur 2025 Year in ReviewAnnual review from the UK's primary government AI safety body, useful for tracking the institutional evolution of national AI safety efforts and the state of frontier model evaluations as of 2025.The UK AI Security Institute (AISI) reviews its 2025 achievements, including publishing the first Frontier AI Trends Report based on two years of testing over 30 frontier AI sys...governanceevaluationred-teamingcapabilities+6Source ↗ - 2025 annual review
Research Outputs
- Frontier AI Trends Report↗🏛️ government★★★★☆UK AI Safety InstituteAISI Frontier AI TrendsPublished by the UK AI Safety Institute (AISI), this report offers an authoritative government perspective on frontier AI capability trends and safety considerations, useful for tracking official assessments of the AI risk landscape.A UK AI Safety Institute government assessment documenting exponential performance improvements across frontier AI systems in multiple domains. The report evaluates emerging cap...capabilitiesai-safetyevaluationred-teaming+5Source ↗ - December 2024 assessment of frontier AI capabilities
- Inspect AI Framework↗🔗 webUK AI Safety Institute's Inspect frameworkInspect is a practical evaluation toolkit from the UK government's AI Safety Institute, relevant to researchers building safety benchmarks or conducting model evaluations; note that current tags like 'interpretability' and 'rlhf' appear mismatched to this resource's actual focus on evaluation infrastructure.Inspect is an open-source framework developed by the UK AI Safety Institute (AISI) for evaluating large language models and AI systems. It provides standardized tools for runnin...ai-safetyevaluationtechnical-safetyred-teaming+4Source ↗ - Open-source evaluation framework documentation
- Inspect Evals↗🔗 webInspect Evals: Community Evaluation Repository for AI ModelsMaintained by the UK AI Safety Institute, this repository provides standardized implementations of major AI benchmarks useful for capability assessment and safety evaluation research.Inspect Evals is a repository of community-contributed AI model evaluations maintained by the UK AI Safety Institute (AISI), featuring implementations of popular benchmarks acro...evaluationcapabilitiesai-safetyred-teaming+2Source ↗ - Collection of 100+ pre-built evaluations
- Early Lessons from Evaluating Frontier AI Systems↗🏛️ government★★★★☆UK AI Safety InstituteUK AI Security Institute's evaluationsPublished by the UK AI Safety Institute (now AI Security Institute), this post is an authoritative government source on frontier model evaluation methodology and represents a key pillar of the UK's approach to pre-deployment AI safety assessment.The UK AI Safety Institute shares early findings and methodology from its evaluations of frontier AI models, covering how they assess potentially dangerous capabilities includin...evaluationred-teaminggovernancepolicy+5Source ↗ - Evaluation methodology insights
Funding Programs
- The Alignment Project↗🏛️ government★★★★☆UK GovernmentThe Alignment ProjectThis is the official homepage for a significant government-backed alignment funding program; useful for researchers seeking grants and for tracking the scale and priorities of state-level AI safety investment as of 2025.The Alignment Project is a major international funding initiative launched in 2025 by the UK AI Security Institute and a broad coalition of government, industry, and philanthrop...alignmentai-safetygovernancepolicy+4Source ↗ - 15 million GBP global alignment research fund
- Grants Overview↗🏛️ government★★★★☆UK AI Safety InstituteAISI Grants Programs OverviewOfficial UK AISI grants page detailing government-funded AI safety research programs; useful for researchers seeking funding history or understanding UK government priorities in AI safety research, though programs are currently closed.The UK AI Safety Institute (AISI) has administered over £15 million in grants across three programs to advance AI safety and security research. Programs include the Alignment Pr...ai-safetyalignmentgovernancepolicy+4Source ↗ - Information on all AISI grant programs
- Systemic Safety Grants↗🏛️ government★★★★☆UK AI Safety InstituteSystemic Safety GrantsOfficial UK AISI announcement of a government-funded research grants programme; relevant to researchers seeking funding for societal AI risk work and to those tracking how governments are operationalizing AI safety research agendas.The UK AI Safety Institute launched a Systemic AI Safety Grants programme offering up to £200,000 to researchers from academia, industry, and civil society. The programme target...ai-safetygovernancepolicydeployment+4Source ↗ - 8.5 million GBP societal resilience program
International Coordination
- The Bletchley Declaration↗🏛️ government★★★★☆UK Governmentgovernment AI policiesA foundational international policy document for AI governance; frequently cited as the first major intergovernmental acknowledgment of catastrophic AI risk, making it highly relevant to tracking the evolution of global AI safety policy.The Bletchley Declaration is a landmark multinational policy agreement signed at the AI Safety Summit 2023, committing participating nations to collaborative efforts on AI safet...governancepolicyai-safetyexistential-risk+3Source ↗ - November 2023 international agreement
- Chair's Summary of AI Safety Summit 2023↗🏛️ government★★★★☆UK GovernmentAI Safety Summit 2023: Chair's Statement (Bletchley Park, 2 November 2023)This is the official chair's summary from the landmark Bletchley Park AI Safety Summit, a key primary source for understanding early international governmental coordination on frontier AI safety and governance frameworks.This is the official UK government Chair's statement from the inaugural AI Safety Summit held at Bletchley Park on 1-2 November 2023. The summit brought together international s...ai-safetygovernancepolicycoordination+3Source ↗ - Summit outcomes
- CSIS: The AI Safety Institute International Network↗🔗 web★★★★☆CSISThe AI Safety Institute International Network: Next StepsPublished by CSIS, this policy analysis is relevant for understanding international efforts to institutionalize AI safety governance through coordinated national safety institutes, particularly following the UK Bletchley Declaration.This CSIS analysis examines the international network of AI Safety Institutes established across multiple countries and provides recommendations for strengthening their coordina...ai-safetygovernancepolicycoordination+3Source ↗ - Analysis of international coordination
- Centre for Future Generations: The AI Safety Institute Network↗🔗 webCentre for Future Generations: The AI Safety Institute NetworkUseful reference for understanding the international landscape of government AI safety institutions, relevant to researchers and policymakers tracking the development of AI governance infrastructure post-2023 AI Safety Summits.This Centre for Future Generations resource analyzes the emerging international network of AI Safety Institutes, examining their membership, mandates, and coordination mechanism...ai-safetygovernancepolicycoordination+2Source ↗ - Comparative analysis
News and Analysis
- UK AI Safety Institute Wikipedia↗📖 reference★★★☆☆WikipediaUK AI Safety Institute WikipediaUseful background reference for understanding the UK's institutional AI safety efforts; relevant to discussions of AI governance, evaluation frameworks, and international policy coordination.Wikipedia article covering the UK AI Safety Institute (AISI), a government body established in 2023 to advance AI safety research and evaluation. It provides an overview of the ...ai-safetygovernancepolicyevaluation+4Source ↗ - Comprehensive background
- Computer Weekly: San Francisco Office Announcement↗🔗 webComputer Weekly: San Francisco Office AnnouncementRelevant to tracking the institutional growth of government-backed AI safety bodies and international coordination efforts between the UK and US on frontier AI oversight.The UK AI Safety Institute (AISI) announced plans to open a branch office in San Francisco, expanding its international presence into a major hub of AI development. This move si...ai-safetygovernancepolicyevaluation+2Source ↗ - May 2024 expansion
- Infosecurity Magazine: AISI Rebrands↗🔗 webInfosecurity Magazine: AISI RebrandsNews coverage of the UK AISI rebrand in early 2025, relevant to those tracking institutional developments in government AI safety bodies and shifts in national AI governance strategy.The UK's AI Safety Institute (AISI) underwent a rebranding as part of a broader government strategy shift in AI policy, reflecting changes in the UK's approach to AI governance ...governancepolicyai-safetygovernment-ai-safety+3Source ↗ - February 2025 name change
- AI Now Statement on Transition↗🔗 web★★★★☆AI Now InstituteAI Now Statement on TransitionPublished by the AI Now Institute, a prominent civil society research organization focused on AI's social implications; relevant to debates about how governments institutionalize AI oversight and what risks get prioritized.The AI Now Institute responds to the UK government's decision to rename and reorient the AI Safety Institute into the AI Security Institute, expressing concern that this shift d...governancepolicygovernment-ai-safetyai-safety+3Source ↗ - Critical perspective on rebrand
References
This CSIS analysis examines the international network of AI Safety Institutes established across multiple countries and provides recommendations for strengthening their coordination, scope, and effectiveness. It addresses how these institutes can better collaborate on technical safety evaluations and policy alignment to address frontier AI risks.
The UK AI Safety Institute shares early findings and methodology from its evaluations of frontier AI models, covering how they assess potentially dangerous capabilities including cybersecurity risks, CBRN threats, and autonomous behavior. The post outlines the AISI's approach to pre-deployment evaluations and the practical challenges encountered when testing leading AI systems.
The Bletchley Declaration is a landmark multinational policy agreement signed at the AI Safety Summit 2023, committing participating nations to collaborative efforts on AI safety while enabling beneficial AI development. It represents one of the first major intergovernmental consensus documents explicitly addressing risks from frontier AI systems, including potential catastrophic and existential harms.
The Alignment Project is a major international funding initiative launched in 2025 by the UK AI Security Institute and a broad coalition of government, industry, and philanthropic partners to accelerate AI alignment research. In its first round, it awarded over £27 million to 60+ projects, with grants up to £1 million supplemented by compute access, expert support, and potential venture capital. The initiative aims to ensure advanced AI systems remain safe, reliable, and under human control.
The UK AI Security Institute (AISI) reviews its 2025 achievements, including publishing the first Frontier AI Trends Report based on two years of testing over 30 frontier AI systems. Key advances include deepened evaluation suites across cyber, chem-bio, and alignment domains, plus pioneering work on sandbagging detection, self-replication benchmarks, and AI-enabled persuasion research published in Science.
Inspect Evals is a repository of community-contributed AI model evaluations maintained by the UK AI Safety Institute (AISI), featuring implementations of popular benchmarks across coding, reasoning, safety, and agent capabilities. Evaluations can be installed and run with a single command against any model, covering domains from Python coding to cybersecurity to scientific reasoning.
The UK government's foundational document introducing the AI Safety Institute (AISI), the first state-backed organization dedicated to advanced AI safety for the public interest. It outlines AISI's mission to minimize surprise from rapid AI advances and develop sociotechnical infrastructure to understand and mitigate AI risks, presented to Parliament in November 2023 by Ian Hogarth.
The UK AI Safety Institute launched a Systemic AI Safety Grants programme offering up to £200,000 to researchers from academia, industry, and civil society. The programme targets broader societal risks from AI deployment—including sector-specific impacts in healthcare, education, and finance, multi-agent interactions, and infrastructure vulnerabilities—beyond individual model capabilities. It aims to map research priorities, develop new methodologies, and inform government policy ahead of anticipated rapid AI adoption over the next 2-5 years.
9AI Safety Summit 2023: Chair's Statement (Bletchley Park, 2 November 2023)UK Government·Government▸
This is the official UK government Chair's statement from the inaugural AI Safety Summit held at Bletchley Park on 1-2 November 2023. The summit brought together international stakeholders to identify next steps for the safe development of frontier AI. It represents a landmark moment in international AI governance coordination, resulting in the Bletchley Declaration.
A UK AI Safety Institute government assessment documenting exponential performance improvements across frontier AI systems in multiple domains. The report evaluates emerging capabilities and associated risks, calling for robust safeguards as systems advance rapidly. It serves as an official benchmark of the current frontier AI landscape from a national safety authority.
The UK's AI Safety Institute (AISI) underwent a rebranding as part of a broader government strategy shift in AI policy, reflecting changes in the UK's approach to AI governance and safety. The article covers the institutional reorganization and its implications for the UK's AI safety landscape.
The UK AI Safety Institute (recently rebranded as the AI Security Institute) is a government body under the Department for Science, Innovation and Technology focused on minimizing risks from rapid and unexpected AI advances. It conducts and publishes safety research, international coordination reports, and policy guidance, while managing grants for systemic AI safety research.
Wikipedia article covering the UK AI Safety Institute (AISI), a government body established in 2023 to advance AI safety research and evaluation. It provides an overview of the institute's mission, structure, key activities such as frontier model evaluations, and its role in international AI safety coordination. The article serves as a reference point for understanding the UK's institutional approach to governing advanced AI.
The UK AI Safety Institute (AISI) has administered over £15 million in grants across three programs to advance AI safety and security research. Programs include the Alignment Project (up to £1M per project), Challenge Fund (up to £200K per project), and Systemic AI Safety Grants focused on societal resilience. All programs are currently closed to new applications.
The AI Now Institute responds to the UK government's decision to rename and reorient the AI Safety Institute into the AI Security Institute, expressing concern that this shift deprioritizes broader societal safety in favor of narrower security framing. The statement argues this transition risks sidelining critical work on AI harms affecting workers, marginalized communities, and democratic institutions. It calls for maintaining robust, independent AI safety oversight.
The UK AI Safety Institute (AISI) announced plans to open a branch office in San Francisco, expanding its international presence into a major hub of AI development. This move signals the UK government's intent to maintain close engagement with leading AI companies and researchers based in Silicon Valley, strengthening its role in global AI safety governance.
The UK AI Safety Institute's fourth progress report documents its rapid institutional growth and key achievements through May 2024, including coordinating the first International Scientific Report on Advanced AI Safety across 30 countries, open-sourcing the Inspect evaluation framework, and expanding to over 30 researchers. The report illustrates how a government body can scale from a single researcher to a significant AI safety evaluation organization within a year.
This Centre for Future Generations resource analyzes the emerging international network of AI Safety Institutes, examining their membership, mandates, and coordination mechanisms. It provides an overview of how national AI safety bodies are structured and how they collaborate across borders to address shared risks from advanced AI systems.
Inspect is an open-source framework developed by the UK AI Safety Institute (AISI) for evaluating large language models and AI systems. It provides standardized tools for running safety evaluations, benchmarks, and red-teaming tasks. The framework enables researchers and developers to assess AI model capabilities and safety properties in a reproducible and extensible way.
The UK AI Safety Institute (AISI) is the UK government's dedicated body for evaluating and mitigating risks from advanced AI systems. It conducts technical safety research, develops evaluation frameworks for frontier AI models, and works with international partners to inform global AI governance and policy.