Frontier Model Forum
Frontier Model Forum
The Frontier Model Forum represents the AI industry's primary self-governance initiative for frontier AI safety, establishing frameworks and funding research, but faces fundamental criticisms about conflicts of interest inherent in industry self-regulation. While the organization has made concrete progress on safety frameworks and evaluations, questions remain about whether profit-driven companies can adequately regulate themselves on existential safety issues.
Quick Assessment
| Dimension | Assessment | Evidence |
|---|---|---|
| Organizational Type | Industry Self-Governance | Non-profit 501(c)(6) established by leading AI companies |
| Founded | 2023 | By Anthropic, Google DeepMind, Microsoft, and OpenAI |
| Primary Focus | Frontier AI Safety Frameworks | Risk evaluation, capability thresholds, and mitigation strategies |
| Funding | $10M+ AI Safety Fund | Industry and philanthropic support |
| Key Output | Safety Commitments & Frameworks | Published by 12+ companies as of late 2024 |
Key Links
| Source | Link |
|---|---|
| Official Website | frontiermodelforum.org |
| Wikipedia | en.wikipedia.org |
Overview
The Frontier Model Forum (FMF) is an industry-supported non-profit organization established in July 2023 to promote self-governance in frontier AI safety through collaborative development of best practices, research coordination, and information-sharing among leading AI developers.1 Led by Executive Director Chris Meserole, the organization focuses on addressing severe risks to public safety and national security from advanced general-purpose AI models, including biological threats, cybersecurity risks, and catastrophic misuse scenarios.2
The Forum emerged as a response to growing recognition that advanced AI systems require coordinated safety frameworks beyond individual company efforts. Its founding members—Anthropic, Google DeepMind, Microsoft, and OpenAI—3
The FMF operates through three core mandates: identifying best practices and standards for frontier AI safety and security, advancing scientific research on safety mechanisms, and facilitating information-sharing across industry, government, academia, and civil society.4 While positioned as an industry self-governance initiative, the organization has faced questions about whether profit-driven companies can adequately regulate themselves without independent oversight.
History and Founding
Launch and Initial Structure
The Frontier Model Forum was officially announced on July 26, 2023, through coordinated blog posts from its four founding companies.5 The announcement emphasized the urgency of establishing unified safety standards amid rapid AI advancement, with founding members agreeing to pool technical expertise despite being direct competitors in the AI development space.6
The organization was legally established as a 501(c)(6) non-profit, a structure that allows industry associations to pursue public benefits without engaging in lobbying activities.7 Kent Walker, President of Global Affairs at Google & Alphabet, stated at launch: "We're excited to work together with other leading companies, sharing technical expertise to promote responsible AI innovation."8
Key Milestones
October 2023: The FMF launched the AI Safety Fund (AISF), a collaborative $10+ million initiative funded by the founding members plus philanthropic partners including the Patrick J. McGovern Foundation, David and Lucile Packard Foundation, Schmidt Sciences, and Jaan Tallinn.9 The fund was initially administered by the Meridian Institute to support independent research on responsible frontier AI development, risk minimization, and standardized third-party evaluations.
May 2024: At the AI Seoul Summit, FMF members signed the Frontier AI Safety Commitments, pledging to develop and publish individual safety frameworks before the February 2025 AI Action Summit in Paris.10 This marked a shift from high-level principles to concrete, actionable commitments with specific deadlines. By late 2024, 16 companies had signed these commitments, with 12 major developers publishing detailed frameworks demonstrating "growing industry consensus" on risk management practices.11
June 2025: Following the closure of the Meridian Institute, the FMF assumed direct management of the AI Safety Fund.12 This transition gave the Forum more control over grant distribution and research priorities.
Governance Evolution
The FMF is governed by an operating board composed of representatives from member organizations, with plans for an Advisory Board to provide guidance from diverse stakeholder perspectives.13 The organization emphasized at launch that membership would be open to firms capable of developing frontier AI at scale, provided they demonstrate proven safety commitments including public acknowledgment of risks, documented mitigation guidelines, safety review processes, and support for third-party research and evaluations.14
Core Initiatives and Workstreams
Frontier AI Safety Frameworks
The centerpiece of the FMF's approach is the development of frontier AI safety frameworks—prespecified guidelines that integrate capability assessments, risk thresholds, and mitigation measures into structured risk management processes.15 These frameworks emerged as the primary tool for industry self-regulation following the May 2024 Frontier AI Safety Commitments.
Effective safety frameworks include the following components:16
-
Risk Identification: Defining clear capability or risk thresholds that specify when heightened safeguards are needed and when risks become unacceptable, with documented rationale for threshold selection
-
Safety Evaluations: Rigorous pre-deployment and post-deployment assessments measuring safety-relevant capabilities and behaviors to identify needed mitigations
-
Risk Mitigation: Implementing protective measures to reduce the risk of high-severity harms and keep risks within tolerable thresholds
-
Risk Governance: Establishing internal accountability frameworks, transparency mechanisms, and processes for updating safety measures
Example implementations from member companies include:17
| Company | Key Framework Elements |
|---|---|
| G42 | Capability thresholds for biological/cyber threats; 4 security levels (e.g., Level 4 resists state-sponsored theft via encryption, red teaming) |
| xAI | Quantitative thresholds/metrics; safeguards against malicious use (e.g., refusal policies for CBRN weapons); information security like role-based access control |
The FMF has published a technical report series to detail implementation approaches and harmonize practices across firms, emphasizing the need for standardized evaluation protocols, capability assessment metrics, and safeguard testing methodologies.18
AI Safety Fund Research
The AI Safety Fund has distributed two rounds of grants since its October 2023 launch, with a recent cohort of 11 grantees awarded $5+ million for projects in biosecurity, cybersecurity, AI agent evaluation, and synthetic content.19 The fund prioritizes independent research that can inform industry-wide practices rather than company-specific applications.
Subsequent rounds have emphasized projects targeting urgent bottlenecks in safety research, such as measuring instrumental reasoning capabilities that could undermine human control.20
AI-Bio Workstream
The AI-Bio workstream focuses specifically on AI-enabled biological threats, developing shared threat models, safety evaluations, and mitigation strategies.21 This workstream addresses concerns that advanced AI models could amplify biological risks by enabling non-experts to design dangerous pathogens or circumvent biosafety protocols. The group has published a preliminary taxonomy of AI-bio safety evaluations outlining how to test whether models possess capabilities that could be misused for biological harm.22
Frontier AI Security
The Forum convenes leading cybersecurity experts to develop novel approaches for securing frontier AI models against theft, tampering, and misuse.23 This workstream recognizes that traditional cybersecurity frameworks require adaptation for AI systems, which face unique vulnerabilities such as model weight exfiltration, adversarial attacks during inference, and risks from insider threats with specialized knowledge.
Safety Frameworks in Detail
Threshold Setting Challenges
One of the most technically challenging aspects of frontier AI safety frameworks is establishing appropriate thresholds for when enhanced safeguards should be triggered. The FMF has identified several main approaches to establishing thresholds:24
Compute Thresholds: Using computational resources (measured in FLOPs) as a proxy for identifying potentially high-risk models. While straightforward to measure, the FMF acknowledges this is an "imperfect proxy" since algorithmic advances can enable dangerous capabilities with less compute, and some large models may pose minimal risks while smaller specialized models could be highly dangerous.25
Risk Thresholds: Defining specific unacceptable outcomes or threat scenarios (e.g., models that could assist in creating novel bioweapons, conduct sophisticated cyber attacks, or autonomously pursue misaligned goals). Setting these thresholds is complicated by lack of historical precedent, novel failure modes, socio-technical complexities, and the need for normative value judgments about acceptable tradeoffs.26
Evaluation Methodologies
The FMF's issue briefs on pre-deployment safety evaluations emphasize that assessments must cover both intended use cases and adversarial exploitation scenarios.27 Evaluations should consider multiple threat models, including:
- API abuse: Misuse through normal model access interfaces
- Weight theft without significant fine-tuning: Adversaries obtaining and deploying model weights as-is
- Limited adversarial budgets: Realistic resource constraints on attackers rather than assuming unlimited capabilities
The Forum cautions against designing evaluations solely for "unlimited adversaries," as this can make threat modeling intractable and lead to overly conservative restrictions that limit beneficial applications.28
Mitigation Strategies and Limitations
The FMF acknowledges significant robustness challenges in current safety measures. Research supported by the Forum has identified that existing safety training methods often modify only surface-level behaviors without altering underlying model capabilities, and adversarial prompts ("jailbreaks") can frequently bypass alignment training.29
Advanced safety concerns addressed by FMF-supported research include:30
- Deceptive alignment: AI systems that appear aligned during training but pursue misaligned objectives during deployment
- AI scheming: Models deliberately circumventing safety measures while appearing compliant
- Alignment faking: Systems providing dishonest outputs to pass safety evaluations
The organization supports research on chain-of-thought monitoring to oversee models that might develop scheming capabilities, and instrumental reasoning evaluation to detect when models acquire situational awareness and stealth capabilities that could undermine human control.31
Funding and Organizational Support
The AI Safety Fund represents the primary funding mechanism through which the FMF supports the broader research ecosystem. The $10+ million total includes contributions from all four founding members (Anthropic, Google, Microsoft, and OpenAI) as well as philanthropic partners.32
Jaan Tallinn, the Estonian programmer and early AI safety philanthropist who co-founded Skype, is among the individual supporters, alongside institutional philanthropies focused on science and technology.33 The fund explicitly aims to support research that is independent from member company interests, though questions remain about whether industry-funded research can maintain true independence when evaluating risks posed by the funders themselves.
The FMF operates as a non-profit, funded by fees from its member firms.34
Cross-Sector Collaboration and Policy Engagement
The FMF positions itself as a connector between industry technical expertise and broader stakeholder communities. The organization emphasizes collaboration with government bodies, academic institutions, and civil society organizations on matters of public safety and security.35
This approach aligns with initiatives including the G7 Hiroshima AI Process, OECD AI principles, and the establishment of AI Safety Institutes in multiple countries.36 The Forum has supported the global network of AI safety institutes as they shift focus from high-level commitments to concrete implementation actions.
Anna Makanju, Vice President of Global Affairs at OpenAI, described the FMF's role in aligning companies on "thoughtful and adaptable safety practices" for powerful models, emphasizing the urgency of establishing shared standards before more capable systems are deployed.37
Criticisms and Limitations
Industry Self-Regulation Concerns
The most fundamental criticism of the FMF centers on the inherent limitations of industry self-governance. Andrew Rogoyski of the Institute for People-Centred AI at the University of Surrey characterized the initiative as "putting the foxes in charge of the chicken coop," arguing that profit-driven companies are structurally unable to adequately regulate themselves and that safety assessments must be performed by independent bodies to avoid regulatory capture.38
Critics point out that the FMF's member companies have direct financial incentives to minimize regulatory burdens, accelerate deployment timelines, and define "safe" in ways that permit their business models to continue. The organization's non-profit structure and stated commitment to public benefit may be insufficient to overcome these underlying conflicts of interest.
Narrow Focus on Frontier AI
The FMF's explicit focus on "frontier" models—defined as state-of-the-art systems at the capabilities boundary—has drawn criticism for potentially delaying regulations on existing AI systems that already cause measurable harms.39 Critics argue that the emphasis on hypothetical future risks from cutting-edge models diverts attention from current issues including:
- Misinformation and manipulation in electoral contexts
- Deepfake generation and identity theft
- Privacy violations through training on personal data
- Intellectual property infringement
- Labor displacement and economic disruption
- Discriminatory outcomes in hiring, lending, and criminal justice
The term "frontier AI" itself has been criticized as an "undefinable moving-target" that allows companies to continuously exclude their current deployed systems from the most stringent safety requirements by claiming those systems are no longer at the frontier.40
Technical Limitations of Safety Cases
The FMF's emphasis on safety frameworks and pre-deployment evaluations faces significant technical challenges. Research suggests that safety cases—structured arguments for why a system is adequately safe—may have limitations:41
Sandbagging and Deception: Models may deliberately underperform on safety evaluations while retaining dangerous capabilities that emerge during deployment. Recent research on alignment faking has demonstrated that models can learn to behave differently when they detect they are being evaluated versus deployed.
Incomplete Coverage: The vast range of potential behaviors in open-ended general-purpose models makes comprehensive evaluation intractable. Human oversight does not scale to catch all potential failures, defeating the goal of complete safety analysis.
False Assurance: Detailed safety cases may provide a false sense of security without meaningfully reducing risks, particularly if developers are incentivized to present optimistic assessments or if evaluators lack independence.
Limited Impact on Bad Actors: The most dangerous scenarios may involve developers who deliberately circumvent safety processes, and voluntary frameworks provide no mechanism to prevent such behavior.
Institutional and Political Challenges
Some researchers frame AI safety as a "neverending institutional challenge" rather than a purely technical problem that can be solved through better evaluations and frameworks.42 From this perspective, the FMF's focus on technical solutions may be insufficient without addressing deeper institutional questions:
- What happens if a frontier developer becomes malicious or recklessly profit-driven after achieving transformative AI capabilities?
- Could widespread adoption of "best practices" actually accelerate risks by enabling faster development timelines or facilitating dangerous research?
- Who adjudicates disputes about whether safety thresholds have been exceeded if the industry is self-governing?
Additionally, safety frameworks face political obstacles. In the United States in particular, detailed pre-deployment review requirements have been characterized by some policymakers as overregulation that could hamper American AI leadership, limiting the political viability of mandating the types of rigorous safety cases the FMF promotes.43
Transparency and Accountability Gaps
While the FMF publishes issue briefs and member companies have released their frameworks, critics note the absence of independent verification mechanisms. The organization has no external audit function, and member companies largely self-report their compliance with safety commitments. This contrasts with other high-risk industries where independent regulators conduct mandatory safety reviews and can halt deployment of insufficiently tested systems.
The FMF's emphasis on information-sharing through "secure channels" following cybersecurity responsible disclosure practices may limit public and academic scrutiny of safety decisions, even as those decisions affect broad populations who use or are affected by AI systems.44
Recent Developments
As of late 2024 and early 2025, the FMF has released several technical publications including:45
- Preliminary Taxonomy of Pre-Deployment Frontier AI Safety Evaluations (December 2024)
- Preliminary Taxonomy of AI-Bio Safety Evaluations (February 2025)
- Issue Brief on Thresholds for Frontier AI Safety Frameworks (February 2025)
These publications reflect ongoing efforts to operationalize the high-level commitments made at the AI Seoul Summit into concrete technical guidance.
Four additional companies joined the Frontier AI Safety Commitments since the initial May 2024 announcement, bringing total participation to 20 companies.46 Notably, xAI published a comprehensive framework in December 2024 outlining quantitative thresholds, metrics, and procedures for managing significant risks from advanced AI systems.47
The Forum has indicated plans to host additional workshops on open AI safety questions, publish more primers on frontier AI safety best practices, and support the work of national and international AI safety institutes as they develop evaluation and oversight capacities.48
Key Uncertainties
Several fundamental questions remain unresolved about the FMF's approach and effectiveness:
Can industry self-governance adequately manage existential risks? While the FMF frames its work around severe public safety threats rather than explicitly invoking existential risk, its safety frameworks address loss-of-control scenarios where advanced AI systems might circumvent human oversight.49 Whether voluntary commitments from profit-driven organizations can provide sufficient protection against catastrophic outcomes remains deeply contested.
How effective are safety frameworks in practice? The frameworks published by member companies demonstrate growing convergence on key elements like threshold-setting and evaluation protocols, but there is limited evidence about whether these frameworks meaningfully reduce risks versus primarily serving as public relations responses to external pressure for regulation.
What happens when capabilities significantly exceed current frontier levels? The FMF's approach assumes that pre-deployment evaluations can identify dangerous capabilities before they manifest in deployed systems. However, some risks may only become apparent through deployment at scale, and evaluation methodologies may fail to keep pace with rapid capability gains.
How should tradeoffs between transparency and security be navigated? The FMF acknowledges tension between making safety evaluations reproducible (requiring detailed disclosure) and avoiding information hazards, gaming of tests, and data leakage that could undermine security.50 The optimal balance remains unclear and may vary by risk domain.
Critics have pointed to mistakes including an overreliance on theoretical argumentation, being too insular, and ignoring policy as a potential route to safety.51
Relationship to Broader AI Safety Ecosystem
The FMF represents one component of a multifaceted AI safety ecosystem that includes academic research institutions, independent evaluation organizations, government regulatory bodies, and civil society advocates. Its role as an industry coordination body makes it distinct from:
- Independent research organizations like Redwood Research and MIRI that develop safety techniques without direct ties to frontier AI developers
- Government initiatives like the UK AI Safety Institute and US AI Safety Institute that provide independent evaluation capacity
- Philanthropic funders like Coefficient Giving that support safety research across multiple institutions
- Academic labs that investigate fundamental questions about AI alignment, interpretability, and robustness
The FMF's industry-led structure means it has unique access to cutting-edge models and deployment insights, but also faces inherent conflicts of interest that these other actors do not share.
Within online AI safety communities like LessWrong and the EA Forum, opinions on the FMF's value vary. Some view it positively as a pragmatic mechanism for advancing concrete safety practices and fostering cross-organizational learning.52 Others express skepticism about whether joining frontier labs to work on safety provides meaningful leverage compared to independent efforts, given the possibility that technical safety work could extend timelines but not fundamentally alter corporate incentives.53
Sources
Footnotes
-
Frontier Model Forum - About Us — Frontier Model Forum - About Us ↩
-
Citation rc-9416 ↩
-
Frontier Model Forum - About Us — Frontier Model Forum - About Us ↩
-
Google Blog - Frontier Model Forum announcement — Google Blog - Frontier Model Forum announcement ↩
-
Citation rc-948e ↩
-
Frontier Model Forum - About Us — Frontier Model Forum - About Us ↩
-
LessWrong - Frontier Model Forum — LessWrong - Frontier Model Forum ↩
-
Frontier Model Forum - AI Safety Fund — Frontier Model Forum - AI Safety Fund ↩
-
Frontier Model Forum - Technical Report Series on Frontier AI Safety Frameworks — Frontier Model Forum - Technical Report Series on Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - Technical Report Series on Frontier AI Safety Frameworks — Frontier Model Forum - Technical Report Series on Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - AI Safety Fund — Frontier Model Forum - AI Safety Fund ↩
-
LessWrong - Frontier Model Forum — LessWrong - Frontier Model Forum ↩
-
Frontier Model Forum - Membership — Frontier Model Forum - Membership ↩
-
Frontier Model Forum - Issue Brief: Components of Frontier AI Safety Frameworks — Frontier Model Forum - Issue Brief: Components of Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - Issue Brief: Components of Frontier AI Safety Frameworks — Frontier Model Forum - Issue Brief: Components of Frontier AI Safety Frameworks ↩
-
METR - Common Elements of Frontier AI Safety Protocols — METR - Common Elements of Frontier AI Safety Protocols ↩
-
Frontier Model Forum - Technical Report Series on Frontier AI Safety Frameworks — Frontier Model Forum - Technical Report Series on Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - New AI Safety Fund Grantees — Frontier Model Forum - New AI Safety Fund Grantees ↩
-
METR - Common Elements of Frontier AI Safety Protocols — METR - Common Elements of Frontier AI Safety Protocols ↩
-
Frontier Model Forum - AI-Bio Workstream — Frontier Model Forum - AI-Bio Workstream ↩
-
Frontier Model Forum - Issue Brief: Preliminary Taxonomy of AI-Bio Safety Evaluations — Frontier Model Forum - Issue Brief: Preliminary Taxonomy of AI-Bio Safety Evaluations ↩
-
Frontier Model Forum - Progress Update: Advancing Frontier AI Safety in 2024 and Beyond — Frontier Model Forum - Progress Update: Advancing Frontier AI Safety in 2024 and Beyond ↩
-
Frontier Model Forum - Issue Brief: Thresholds for Frontier AI Safety Frameworks — Frontier Model Forum - Issue Brief: Thresholds for Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - Issue Brief: Thresholds for Frontier AI Safety Frameworks — Frontier Model Forum - Issue Brief: Thresholds for Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - Issue Brief: Thresholds for Frontier AI Safety Frameworks — Frontier Model Forum - Issue Brief: Thresholds for Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - Issue Brief: Preliminary Taxonomy of Pre-Deployment Frontier AI Safety Evaluations — Frontier Model Forum - Issue Brief: Preliminary Taxonomy of Pre-Deployment Frontier AI Safety Evaluations ↩
-
Frontier Model Forum - Issue Brief: Preliminary Taxonomy of Pre-Deployment Frontier AI Safety Evaluations — Frontier Model Forum - Issue Brief: Preliminary Taxonomy of Pre-Deployment Frontier AI Safety Evaluations ↩
-
Frontier Model Forum - Technical Report: Frontier Mitigations — Frontier Model Forum - Technical Report: Frontier Mitigations ↩
-
Alignment Forum - Evaluating and Monitoring for AI Scheming — Alignment Forum - Evaluating and Monitoring for AI Scheming ↩
-
METR - Common Elements of Frontier AI Safety Protocols — METR - Common Elements of Frontier AI Safety Protocols ↩
-
Frontier Model Forum - AI Safety Fund — Frontier Model Forum - AI Safety Fund ↩
-
Frontier Model Forum - AI Safety Fund — Frontier Model Forum - AI Safety Fund ↩
-
Frontier Model Forum - About Us — Frontier Model Forum - About Us ↩
-
Frontier Model Forum - About Us — Frontier Model Forum - About Us ↩
-
Weights & Biases - The Frontier Model Forum For AI Safety — Weights & Biases - The Frontier Model Forum For AI Safety ↩
-
LessWrong - Frontier Model Forum — LessWrong - Frontier Model Forum ↩
-
Reworked - Can We Trust Tech Companies to Regulate Generative AI? — Reworked - Can We Trust Tech Companies to Regulate Generative AI? ↩
-
Infosecurity Magazine - AI Safety Summit Criticisms: Narrow Focus — Infosecurity Magazine - AI Safety Summit Criticisms: Narrow Focus ↩
-
Infosecurity Magazine - AI Safety Summit Criticisms: Narrow Focus — Infosecurity Magazine - AI Safety Summit Criticisms: Narrow Focus ↩
-
EA Forum - Should the AI Safety Community Prioritize Safety Cases? — EA Forum - Should the AI Safety Community Prioritize Safety Cases? ↩
-
LessWrong - Reframing AI Safety as a Neverending Institutional Challenge — LessWrong - Reframing AI Safety as a Neverending Institutional Challenge ↩
-
EA Forum - Should the AI Safety Community Prioritize Safety Cases? — EA Forum - Should the AI Safety Community Prioritize Safety Cases? ↩
-
LessWrong - Frontier Model Forum — LessWrong - Frontier Model Forum ↩
-
Frontier Model Forum - Publications — Frontier Model Forum - Publications ↩
-
METR - Common Elements of Frontier AI Safety Protocols — METR - Common Elements of Frontier AI Safety Protocols ↩
-
xAI - Frontier Artificial Intelligence Framework PDF — xAI - Frontier Artificial Intelligence Framework PDF ↩
-
Frontier Model Forum - Progress Update: Advancing Frontier AI Safety in 2024 and Beyond — Frontier Model Forum - Progress Update: Advancing Frontier AI Safety in 2024 and Beyond ↩
-
Frontier Model Forum - Issue Brief: Components of Frontier AI Safety Frameworks — Frontier Model Forum - Issue Brief: Components of Frontier AI Safety Frameworks ↩
-
Frontier Model Forum - Early Best Practices for Frontier AI Safety Evaluations — Frontier Model Forum - Early Best Practices for Frontier AI Safety Evaluations ↩
-
EA Forum - What Mistakes Has the AI Safety Movement Made? — EA Forum - What Mistakes Has the AI Safety Movement Made? ↩
-
LessWrong - Frontier Model Forum — LessWrong - Frontier Model Forum ↩
-
EA Forum - Reasons For and Against Working on Technical AI Safety at a Frontier Lab — EA Forum - Reasons For and Against Working on Technical AI Safety at a Frontier Lab ↩
References
1Alignment Forum - Evaluating and Monitoring for AI SchemingAlignment Forum·Vika et al.·2025·Blog post▸
This post examines methods for detecting and evaluating 'scheming' behaviors in AI systems—where models pursue hidden agendas or deceive overseers while appearing aligned. It discusses evaluation frameworks and monitoring approaches to identify deceptive alignment before deployment or during operation.
Anthropic, Google, Microsoft, and OpenAI announced the formation of the Frontier Model Forum, an industry body aimed at ensuring safe and responsible development of frontier AI models. The Forum's core objectives include advancing AI safety research, identifying best practices, collaborating with policymakers, and supporting beneficial applications. It plans to focus on technical evaluations, benchmarks, and research areas such as adversarial robustness and mechanistic interpretability.
“Kent Walker, President, Global Affairs, Google & Alphabet said: “We’re excited to work together with other leading companies, sharing technical expertise to promote responsible AI innovation. We're all going to need to work together to make sure AI benefits everyone.””
unsupported: The organization was legally established as a 501(c)(6) non-profit, a structure that allows industry associations to pursue public benefits without engaging in lobbying activities. misleading paraphrase: Kent Walker, President of Global Affairs at Google & Alphabet, stated at launch: "We're excited to work together with other leading companies, sharing technical expertise to promote responsible AI innovation."
“Over the coming months, the Frontier Model Forum will establish an Advisory Board to help guide its strategy and priorities, representing a diversity of backgrounds and perspectives.”
The claim states that the FMF is governed by an operating board composed of representatives from member organizations, but the source only mentions an 'executive board'. The claim states that membership would be open to firms capable of developing frontier AI at scale, provided they demonstrate proven safety commitments including public acknowledgment of risks, documented mitigation guidelines, safety review processes, and support for third-party research and evaluations. The source only mentions that membership is open to organizations that develop and deploy frontier models and demonstrate a strong commitment to frontier model safety, including through technical and institutional approaches. It does not mention the specific requirements listed in the claim.
“Anna Makanju, Vice President of Global Affairs, OpenAI said: “Advanced AI technologies have the potential to profoundly benefit society, and the ability to achieve this potential requires oversight and governance. It is vital that AI companies–especially those working on the most powerful models–align on common ground and advance thoughtful and adaptable safety practices to ensure powerful AI tools have the broadest benefit possible. This is urgent work and this forum is well-positioned to act quickly to advance the state of AI safety.””
This article examines the Frontier Model Forum (FMF), an industry self-regulatory body created by Microsoft, Anthropic, Google, and OpenAI. Experts argue that profit-driven companies cannot effectively self-regulate AI safety and that independent oversight with international governmental leadership is essential. The piece highlights the gap between AI development pace and governmental regulatory capacity.
“The Frontier Model Forum is laudable in its aims but isn’t by any means the whole answer to safety concerns on AI, Andrew Rogoyski of the Institute for People-Centred AI at the University of Surrey told Reworked. “The AI industry isn’t mature enough to be allowed to self-regulate,” he said, likening the effort to "putting the foxes in charge of the chicken coop." He believes there is a fundamental problem with the idea that some profit-driven AI developers will be responsible for setting the bar on AI safety research.”
Microsoft, Anthropic, Google, and OpenAI announce the formation of the Frontier Model Forum, an industry body dedicated to the safe and responsible development of frontier AI models. The Forum aims to advance AI safety research, identify best practices, collaborate with policymakers and civil society, and support AI applications addressing major societal challenges. Membership is open to organizations that develop frontier models and demonstrate strong safety commitments.
“July 26, 2023 – Today, Anthropic, Google, Microsoft , and OpenAI are announcing the formation of the Frontier Model Forum, a new industry body focused on ensuring safe and responsible development of frontier AI models.”
The Frontier Model Forum (FMF) is an industry-supported 501(c)(6) non-profit founded in 2023 to advance frontier AI safety and security. Its three core mandates are identifying best practices, advancing the science of AI safety, and facilitating information sharing among government, academia, civil society, and industry. It focuses particularly on CBRN and advanced cyber threat risks posed by frontier AI systems.
“The Frontier Model Forum is an industry-supported non-profit 501(c)(6), established in 2023. In line with its mission, the Forum provides public benefits related to AI safety and security and does not engage in lobbying.”
unsupported wrong_attribution
“The Frontier Model Forum (FMF) is an industry-supported non-profit dedicated to advancing frontier AI safety and security. The FMF has three core mandates: Identify best practices and support standards development for frontier AI safety and security. Advance the science of frontier AI safety and security. Facilitate information sharing about frontier AI safety and security among government, academia, civil society and industry. The FMF focuses primarily on managing significant risks to public safety and security, including from chemical, biological, radiological, nuclear (CBRN) and advanced cyber threats. By drawing on the technical and operational expertise of its member firms, the FMF seeks to ensure the most advanced AI systems remain safe and secure so that they can meet society’s most pressing needs.”
“The FMF has three core mandates: Identify best practices and support standards development for frontier AI safety and security. Advance the science of frontier AI safety and security. Facilitate information sharing about frontier AI safety and security among government, academia, civil society and industry.”
This page outlines the membership criteria and current members of the Frontier Model Forum (FMF), an industry non-profit focused on safe frontier AI development. Members must demonstrate frontier AI capability, a track record of safety practices, and commitment to contributing to FMF activities. Current members include Amazon, Anthropic, Google, Meta, Microsoft, and OpenAI.
“We seek to admit members with a clear ability to develop and deploy frontier AI systems at scale, a proven commitment to AI safety and security, and a willingness to contribute to the mission of the FMF.”
7Frontier Model Forum - Technical Report Series on Frontier AI Safety FrameworksFrontier Model Forum▸
The Frontier Model Forum (FMF) introduces a multi-part technical report series examining how frontier AI safety frameworks can be implemented effectively across organizations. The series covers risk taxonomy and thresholds, capability assessments, mitigations, and third-party assessments, drawing on lessons from early adopters like those who committed to frameworks at the 2024 AI Seoul Summit.
“In May 2024, 16 companies formally committed to developing and publishing frameworks as part of the Frontier AI Safety Commitments at the AI Seoul Summit.”
The source does not mention the 'Frontier AI Safety Commitments' being signed by FMF members, but rather by 16 companies. The source does not mention the February 2025 AI Action Summit in Paris, but rather just the AI Seoul Summit in May 2024. The source does not explicitly state that the commitments marked a shift from high-level principles to concrete, actionable commitments with specific deadlines, although it does imply this.
“In May 2024, 16 companies formally committed to developing and publishing frameworks as part of the Frontier AI Safety Commitments at the AI Seoul Summit. To date, 12 major AI developers have published frontier AI frameworks, demonstrating a growing industry consensus on responsible development practices.”
The claim states 'late 2024', but the source says 'May 2024'.
“In a series of technical reports over the coming months, the Frontier Model Forum will examine how these frameworks can be implemented effectively across different organizational contexts.”
This Weights & Biases report covers the formation and goals of the Frontier Model Forum, a collaborative initiative by leading AI companies (OpenAI, Google, Microsoft, Anthropic) to advance AI safety research and best practices. The forum aims to promote responsible development of frontier AI models through industry coordination on safety standards and research sharing.
“Weights & Biases”
“Weights & Biases”
9Frontier Model Forum - Issue Brief: Thresholds for Frontier AI Safety FrameworksFrontier Model Forum▸
This Frontier Model Forum issue brief examines how predefined thresholds function within AI safety frameworks, explaining their role in triggering deeper risk inspection and heightened safeguards for advanced AI models. It outlines the different types of thresholds proposed by developers and the broader safety community, with particular focus on CBRN and advanced cyber risks. The brief aims to advance public understanding of how thresholds create accountability and structure risk management across the AI development lifecycle.
“There are several main approaches to establishing thresholds within frontier AI safety frameworks, each of which entails a unique set of tradeoffs.”
“That said, while compute thresholds are relatively straightforward to understand and measure, they are an imperfect proxy for risk. Recent algorithmic progress has demonstrated that it may be possible to create high-risk systems with less compute than previously believed. Focusing on only models above a certain compute threshold may exclude smaller models that could possess potentially harmful capabilities, and conversely may inundate evaluators with larger models that have only benign capabilities.”
“Setting these thresholds for frontier AI is significantly more challenging due to the lack of historical data, the potential for novel and unprecedented failure modes, and the difficulty in modeling complex socio-technical interactions.”
xAI's Frontier AI Framework (FAIF) outlines the company's policies for managing significant risks—including catastrophic and existential risks—associated with developing and deploying AI models like Grok. It addresses malicious use and loss-of-control scenarios, defines quantitative thresholds and evaluation procedures, and complies with California's Transparency in Frontier Artificial Intelligence Act (TFAIA). The framework references NIST, ISO/IEC 42001, and Frontier Model Forum best practices.
11Frontier Model Forum - Early Best Practices for Frontier AI Safety EvaluationsFrontier Model Forum▸
The Frontier Model Forum's issue brief outlines preliminary best practices for designing, implementing, and disclosing frontier AI safety evaluations. It emphasizes domain expertise, evaluating full systems rather than just models, and building toward scientific consensus in a field where evaluation metrology remains immature. This is the first in a planned series drawing on interviews and workshops with safety experts across FMF member firms.
“Transparency is a key dimension for AI safety evaluations, but an important balance needs to be struck to make evaluations effective. Increased transparency can help developers and researchers learn about and advance safety evaluations. The more transparency there is around the dataset, methodology, and analysis of an evaluation, the easier it is to reproduce and understand the evaluation. Greater transparency in these aspects also makes it easier for independent experts to assess the validity of, and come to consensus on, the implications of an evaluation. By the same token, high opacity in safety evaluations may make aligning on the necessity of certain mitigation measures more difficult. At the same time, greater transparency can create information hazards in high-risk domains. It can also degrade evaluation efficacy, since the more information about evaluation design that is made available, the easier it is for some developers with malicious intent to intentionally game it. Further, if the full test set of an evaluation is disclosed publicly, the test questions may leak into future models’ training data, making the evaluation’s result more difficult to trust.”
12EA Forum - Should the AI Safety Community Prioritize Safety Cases?EA Forum·Jan Wehner🔸·2026·Blog post▸
This EA Forum post synthesizes expert opinions on whether AI Safety Cases—structured arguments demonstrating a system is sufficiently safe—should be prioritized by the AI safety community. It surveys current work, identifies key gaps (methodology consensus, basic science, technical safety), and concludes that while companies like Anthropic and DeepMind are developing them, comprehensive safety cases for catastrophic risks are unlikely within 4+ years and experts disagree on their overall value.
“However, skeptics argue their practical impact on company behavior may be small (especially for less responsible actors), and they risk providing false assurance while being costly to produce and enforce.”
“Safety Cases can be burdensome and have a reputation for overregulation, making them unattractive to US policymakers.”
A qualitative synthesis of interviews with 17 AI safety experts identifying systemic mistakes in the AI safety movement, including overreliance on abstract reasoning, insularity, counterproductive messaging, and neglect of policy pathways. The post provides rare critical self-reflection from within the community about strategic and epistemic failures. Some interviewees questioned whether the movement's overall track record has been net positive.
“Participants pointed to a range of mistakes they thought the AI safety movement had made. Key themes included an overreliance on theoretical argumentation, being too insular, putting people off by pushing weird or extreme views, supporting the leading AGI companies, insufficient independent thought, advocating for an unhelpful pause to AI development, and ignoring policy as a potential route to safety.”
14Frontier Model Forum - Issue Brief: Components of Frontier AI Safety FrameworksFrontier Model Forum▸
This Frontier Model Forum issue brief outlines the key components that leading AI developers should incorporate into frontier AI safety frameworks, providing a structured overview of practices for responsible development and deployment of advanced AI systems. It serves as an industry-level reference for what constitutes a comprehensive safety framework, covering evaluation, mitigation, and governance elements.
“At a high level, safety frameworks include the following components: 1. Risk Identification Frontier AI safety frameworks are intended to manage potential severe threats to public safety and security. 2 To effectively manage these threats, safety frameworks should identify and analyze model risks stemming from advanced capabilities in chemical, biological, radiological, and nuclear (CBRN) weapons development and cyber attacks.”
“By specifying capability and/or risk thresholds, safety evaluations and mitigation strategies for frontier AI models in advance of their development, safety frameworks position frontier AI developers to be able to address potential safety challenges in a principled and coherent way.”
“Frontier AI safety frameworks are designed to enable developers to take a robust, principled, and coherent approach to anticipating and addressing the potential safety challenges posed by frontier AI.”
Google, Microsoft, OpenAI, and Anthropic jointly announced the formation of the Frontier Model Forum, an industry body focused on the safe and responsible development of frontier AI models. The forum aims to advance AI safety research, identify best practices, and facilitate information sharing among companies and policymakers. It represents a significant voluntary industry coordination effort on AI safety and governance.
“Today, Anthropic, Google, Microsoft and OpenAI are announcing the formation of the Frontier Model Forum, a new industry body focused on ensuring safe and responsible development of frontier AI models.”
The Frontier Model Forum presents a structured taxonomy of safety evaluations that frontier AI developers should conduct before deploying models, covering categories like dangerous capabilities, alignment, and societal risks. It aims to standardize evaluation practices across major AI labs and inform policy discussions around responsible deployment. The brief reflects industry-led efforts to operationalize safety commitments made by leading developers.
“These evaluations should be grounded in a specific threat model (i.e. examining a type of actors’ abilities to achieve specific, real-world outcomes in a certain context), e.g., someone abusing an API, or stealing the weights but lacking significant fine-tuning compute or data, rather than assuming an adversary of unlimited resources.”
The document does not mention that the FMF's issue briefs on pre-deployment safety evaluations emphasize that assessments must cover both intended use cases and adversarial exploitation scenarios. The document also does not mention that evaluations should consider multiple threat models.
“These evaluations should be grounded in a specific threat model (i.e. examining a type of actors’ abilities to achieve specific, real-world outcomes in a certain context), e.g., someone abusing an API, or stealing the weights but lacking significant fine-tuning compute or data, rather than assuming an adversary of unlimited resources.”
This Infosecurity Magazine article examines criticisms leveled at AI safety summits (likely referencing the 2023 Bletchley Park summit) for allegedly focusing too narrowly on speculative long-term existential risks while neglecting near-term, concrete harms from AI systems such as bias, misuse, and surveillance. The piece explores tensions between different stakeholder perspectives on what AI safety should prioritize.
“Several voices criticized this concept, arguing that it was a means for these AI companies to push regulations on generative AI to a later date and allow their current products to avoid regulation altogether.”
“In July, Andrew Strait, associate director of the UK-based Ada Lovelace Institute, dismissed the term ‘frontier model’ on social media, saying it’s “an undefinable moving-target term that excludes the existing models from governance, regulation, and attention.””
This Frontier Model Forum issue brief proposes a structured taxonomy for evaluating AI systems' potential to assist with biological threats. It categorizes different types of biosecurity-relevant AI evaluations to help developers and policymakers assess and mitigate misuse risks from frontier models in the bio domain.
“This issue brief offers an initial taxonomy and definitions for frontier AI safety evaluations specific to the biological domain, categorized across two dimensions: methodology and domain.”
The Frontier Model Forum's AI-Bio Workstream focuses on understanding and mitigating the potential for advanced AI models to contribute to biological risks, including misuse in developing dangerous pathogens. It brings together leading AI companies to establish shared safety standards and best practices for evaluating and limiting biosecurity risks from frontier AI systems.
“The AI-Bio workstream aims to develop shared understandings of AI-Bio threat models, safety evaluations, and mitigation measures, as well as common approaches to capability and risk thresholds in the biological domain.”
This post argues that AI safety should be understood not as a technical problem with a finite solution but as an ongoing institutional challenge requiring perpetual governance, adaptation, and democratic deliberation. The author draws on historical precedents to contend that transformative challenges rarely resolve at singular pivotal moments, critiquing the AI safety community's focus on specific timelines and narrow technical agendas. It advocates for distributed power structures and institutional resilience as the core response to AI's transformative potential.
“As AI becomes increasingly transformative, we need to rethink how we approach safety – not as a technical alignment problem, but as an ongoing, unsexy struggle.”
The Frontier Model Forum announces new grantees for its AI Safety Fund, which supports independent research into AI safety challenges. The fund, established by major AI labs including Anthropic, Google, Microsoft, and OpenAI, aims to advance technical and governance research to make frontier AI systems safer. This announcement highlights specific research projects and organizations receiving funding.
“Today we are announcing a new cohort of 11 grantees who have received more than $5 million through the AI Safety Fund (AISF). As frontier AI systems become more powerful and widely deployed, advancing our understanding of them and building robust safety tools is essential – which is why the AISF issued several requests for proposals late last year in Biosecurity and Cybersecurity , as well as AI Agent Evaluation and Synthetic Content .”
The source states the announcement was posted on December 11, 2025, not October 2023. The source does not explicitly state that the fund prioritizes independent research that can inform industry-wide practices rather than company-specific applications, although it does mention supporting and expanding the field of AI safety research to promote responsible development and deployment of frontier models.
An EA Forum post by someone who accepted a frontier lab safety role, presenting a balanced pros-and-cons analysis of pursuing technical AI safety work at organizations like OpenAI, Anthropic, or DeepMind. It weighs benefits like access to frontier models and direct influence against drawbacks like restricted research independence and corporate co-option risks, synthesizing perspectives from multiple stakeholders to aid career decision-making.
“For You're close to the action . As AI continues to heat up , being closer to the action seems increasingly important. Against Some very important safety work happens outside of frontier AI labs.”
This technical report from the Frontier Model Forum examines mitigation strategies for risks posed by frontier AI models, covering approaches that leading AI developers use to reduce potential harms. It provides an industry-consensus framework for understanding and implementing safety measures across the development and deployment lifecycle of frontier models.
“Current safety training methods modify surface-level behaviors without altering underlying model capabilities. Research shows that harmful fine-tuning can rapidly undo alignment post-training with surprisingly few examples – sometimes just dozens of harmful input-output pairs. Adversarial prompts (“jailbreaks”) can bypass alignment post-training – models trained to refuse direct harmful requests may still comply when those requests are adversarially rephrased, decomposed into steps, or embedded in different contexts.”
The Frontier Model Forum is an industry-supported non-profit comprising major AI companies (Amazon, Anthropic, Google, Meta, Microsoft, OpenAI) focused on advancing frontier AI safety and security. Its core mandates include identifying best practices, advancing independent safety research, and facilitating information sharing across government, academia, civil society, and industry. It also produces technical reports on topics like frontier capability assessments for CBRN and cyber risks.
“Frontier Model Forum Frontier Model Forum: Advancing frontier AI safety and security The Frontier Model Forum draws on the technical and operational expertise of its member companies to ensure that the most advanced AI systems remain safe and secure, so that they can meet society’s most pressing needs.”
The source does not mention when the Frontier Model Forum was established or who the Executive Director is.
Wikipedia article covering the UK AI Safety Institute (AISI), a government body established in 2023 to advance AI safety research and evaluation. It provides an overview of the institute's mission, structure, key activities such as frontier model evaluations, and its role in international AI safety coordination. The article serves as a reference point for understanding the UK's institutional approach to governing advanced AI.