UK AI Safety Institute

The UK AI Safety Institute launched a Systemic AI Safety Grants programme offering up to £200,000 to researchers from academia, industry, and civil society. The programme targets broader societal risks from AI deployment—including sector-specific impacts in healthcare, education, and finance, multi-agent interactions, and infrastructure vulnerabilities—beyond individual model capabilities. It aims to map research priorities, develop new methodologies, and inform government policy ahead of anticipated rapid AI adoption over the next 2-5 years.

A UK AI Safety Institute government assessment documenting exponential performance improvements across frontier AI systems in multiple domains. The report evaluates emerging capabilities and associated risks, calling for robust safeguards as systems advance rapidly. It serves as an official benchmark of the current frontier AI landscape from a national safety authority.

The UK AI Safety Institute (AISI) has administered over £15 million in grants across three programs to advance AI safety and security research. Programs include the Alignment Project (up to £1M per project), Challenge Fund (up to £200K per project), and Systemic AI Safety Grants focused on societal resilience. All programs are currently closed to new applications.

The UK AI Safety Institute announced the first 60 grantees of the Alignment Project, a global funding initiative now totaling £27 million with partners including OpenAI, Microsoft, and Anthropic. The program received 800+ applications from 466 institutions across 42 countries, selecting projects spanning mathematics, learning theory, economics, and cognitive science. This represents a significant institutional commitment to scaling alignment research infrastructure globally.

The UK AI Security Institute (AISI) outlines its technical research strategy to close the growing gap between AI capabilities and available safety mitigations. Research is organized into two pillars: Safeguards Analysis (countering misuse by adversarial actors) and Control & Alignment (preventing loss of control of highly capable systems). AISI is actively hiring researchers and offering up to £200,000 in Challenge Fund grants to academics and non-profits to accelerate progress on critical safety challenges.

The UK AI Security Institute's inaugural Frontier AI Trends Report synthesizes evaluations of 30+ frontier AI models to document rapid capability gains across chemistry, biology, and cybersecurity domains. Key findings include models surpassing PhD-level expertise in CBRN fields, cyber task success rates rising from 9% to 50% in under two years, persistent jailbreak vulnerabilities, and growing AI autonomy. The report highlights a dangerous gap between capability advancement and policy adaptation.

The UK and US AI Safety Institutes conducted a joint pre-deployment evaluation of Anthropic's upgraded Claude 3.5 Sonnet, assessing biological capabilities, cyber capabilities, software/AI development, and safeguard efficacy. The evaluation used multiple methodologies including red teaming and agent tasks, benchmarking against prior Claude 3.5 Sonnet, GPT-4o, and o1-preview. This represents an early example of government-led pre-deployment safety testing of frontier AI models.

The UK AI Security Institute (AISI) reviews its 2025 achievements, including publishing the first Frontier AI Trends Report based on two years of testing over 30 frontier AI systems. Key advances include deepened evaluation suites across cyber, chem-bio, and alignment domains, plus pioneering work on sandbagging detection, self-replication benchmarks, and AI-enabled persuasion research published in Science.

The US and UK AI Safety Institutes jointly conducted a pre-deployment safety evaluation of OpenAI's o1 reasoning model, assessing its capabilities in cyber, biological, and software development domains. The evaluation benchmarked o1 against reference models to identify potential risks before public release. This represents an early example of government-led pre-deployment AI safety testing through formal institute collaboration.

The UK AI Safety Institute (AISI) is the UK government's dedicated body for evaluating and mitigating risks from advanced AI systems. It conducts technical safety research, develops evaluation frameworks for frontier AI models, and works with international partners to inform global AI governance and policy.

The UK AI Safety Institute evaluated five anonymized large language models across cyber, chemical/biological, agent, and jailbreak dimensions. Key findings show models exhibit PhD-level CBRN knowledge, limited but real cybersecurity capabilities, nascent agentic behavior, and widespread vulnerability to jailbreaks—providing an early empirical baseline for frontier model risk assessment.

The UK AI Safety Institute shares early findings and methodology from its evaluations of frontier AI models, covering how they assess potentially dangerous capabilities including cybersecurity risks, CBRN threats, and autonomous behavior. The post outlines the AISI's approach to pre-deployment evaluations and the practical challenges encountered when testing leading AI systems.