Skip to content
Longterm Wiki

METR (Model Evaluation & Threat Research)

reference
Wikipedia·en.wikipedia.org/wiki/METR

Credibility Rating

3/5
Good(3)

Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.

Rating inherited from publication venue: Wikipedia

METR is a key organization in AI safety evaluations; understanding their work is important for anyone studying how frontier labs assess and manage dangerous capability thresholds before deploying advanced models.

Metadata

Importance: 72/100wiki pagereference

Summary

METR is an AI safety organization focused on evaluating frontier AI models for dangerous capabilities, particularly autonomous replication and adaptation (ARA) abilities. They develop standardized evaluations to assess whether AI systems pose catastrophic risks, and work with leading AI labs to conduct pre-deployment safety testing. Their work informs responsible scaling policies and deployment decisions.

Key Points

  • •METR develops evaluations for dangerous AI capabilities, especially autonomous replication and adaptation (ARA) that could pose catastrophic risks
  • •Conducts third-party evaluations of frontier models for major AI labs including Anthropic, OpenAI, and Google DeepMind
  • •Focuses on assessing whether AI agents can autonomously acquire resources, evade shutdown, or self-replicate without human oversight
  • •Their evaluation frameworks inform responsible scaling policies (RSPs) used by frontier AI labs as safety checkpoints
  • •Originally spun out of ARC (Alignment Research Center) as ARC Evals before rebranding to METR

Cited by 1 page

PageTypeQuality
METROrganization66.0

Cached Content Preview

HTTP 200Fetched Apr 9, 202612 KB
METR - Wikipedia 

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Jump to content 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 From Wikipedia, the free encyclopedia 
 
 
 
 
 
 AI model evaluation nonprofit 
 

 METR Formation 2022 &#59; 4 years ago  ( 2022 ) Founder Beth Barnes Type Nonprofit research institute Legal status 501(c)(3) tax exempt charity Purpose AI safety research and model evaluation Location Berkeley, California 
 Website metr .org 
 Model Evaluation and Threat Research ( METR ) ( MEE -tər ), is a nonprofit research institute , based in Berkeley, California , [ 1 ] that evaluates frontier AI models' capabilities to carry out long-horizon, agentic tasks that some researchers argue could pose catastrophic risks to society . [ 2 ] [ 3 ] They have worked with leading AI companies to conduct pre-deployment model evaluations and contribute to system cards, including OpenAI 's o3 , o4-mini , GPT-4o and GPT-4.5 , and Anthropic 's Claude models. [ 3 ] [ 4 ] [ 5 ] [ 6 ] [ 7 ] 

 METR's CEO and founder is Beth Barnes, a former alignment researcher at OpenAI who left in 2022 to form ARC Evals, the evaluation division of Paul Christiano 's Alignment Research Center . In December 2023, ARC Evals was then spun off into an independent 501(c)(3) nonprofit and renamed METR. [ 8 ] [ 9 ] [ 10 ] 

 
 Research

 [ edit ] 
 A substantial amount of METR's research is focused on evaluating the capabilities of AI systems to conduct research and development of AI systems themselves , including RE-Bench, a benchmark designed to test whether AIs can "solve research engineering tasks and accelerate AI R&D". [ 11 ] [ 12 ] 

 Doubling time estimates

 [ edit ] 
 A graph showing that the length of tasks frontier models are capable of executing at a 50% success rate doubled every 7 months from 2019 to 2024. The shaded region represents a 95% confidence interval . [ 13 ] 
 In March 2025, METR published a paper noting that the length of software engineering tasks that the leading AI model could complete had a doubling time of around 7 months between 2019 and 2024. [ 14 ] 

 In January 2026, METR has released a new version of their time horizon estimates model (Time Horizon 1.1). According to their new model the rate of progress of AI capabilities has increased since 2023. They now estimate that the post-2023 doubling-time is 130.8 days (4.3 months). Progress is thus estimated to be 20% more rapid. [ 15 ] 

 Time horizon measurements

 [ edit ] 
 METR releases a "task-completion time horizon" for analysed AI models. This measures the "task duration (measured by human expert completion time) at which an AI agent is predicted to succeed with a given level of reliability." [ 16 ] They release it in two variants: The 50%-time horizon, which gives the task duration at which 

... (truncated, 12 KB total)
Resource ID: ab9cc01cf367fd79 | Stable ID: sid_Fsesbh9gXk