Skip to content
Longterm Wiki
Back

MIRI Research Overview

web
MIRI·intelligence.org/research/

Credibility Rating

3/5
Good(3)

Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.

Rating inherited from publication venue: MIRI

MIRI is one of the oldest AI safety organizations; this page serves as an entry point to their research agenda and is relevant for understanding the agent-foundations approach to alignment and long-term existential risk from advanced AI.

Metadata

Importance: 55/100homepage

Summary

This page outlines the Machine Intelligence Research Institute's (MIRI) research agenda and open positions, focusing on their work on technical AI safety and alignment. MIRI pursues foundational mathematical research aimed at ensuring advanced AI systems behave as intended, with a focus on long-term existential risk reduction.

Key Points

  • •MIRI focuses on foundational technical research to ensure advanced AI systems are safe and aligned with human values.
  • •Research areas historically include decision theory, logical uncertainty, and agent foundations.
  • •MIRI operates from an existential risk perspective, treating misaligned superintelligent AI as a critical threat.
  • •The organization recruits researchers with strong mathematical and formal reasoning backgrounds.
  • •Work is oriented toward long-horizon problems that may not have near-term empirical testability.

Cited by 1 page

PageTypeQuality
AI Value Lock-inRisk64.0

Cached Content Preview

HTTP 200Fetched Mar 20, 20268 KB
[Skip to content](https://intelligence.org/research/#content)

# Research

## Technical Governance

MIRI’s current focus is on attempting to halt the development of increasingly general AI models, via discussions with policymakers about the extreme risks artificial superintelligence poses. Our **technical governance** research explores the technical questions that bear on these regulatory and policy goals.

[Learn More About Our Technical Governance Research](https://techgov.intelligence.org/)

## Technical Governance Papers

![](https://intelligence.org/wp-content/uploads/2025/09/1757100105452-4f50b0b9-7f5c-4e7c-8ebd-1d398b55f8eb_1.jpg)

## AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions

This AI governance research agenda lays out our view of the strategic landscape and actionable research questions that, if answered, would provide important insight on how to reduce catastrophic and extinction risks from AI.

[Read more](https://intelligence.org/wp-content/uploads/2025/05/AI-Governance-to-Avoid-Extinction.pdf)

![](https://intelligence.org/wp-content/uploads/2024/12/https_arxiv.org_pdf_2412.08653_page-0001-1.jpg)

## What AI evaluations for preventing catastrophic risks can and cannot do

AI evaluations are an important component of the AI governance toolkit, underlying current approaches to safety cases for preventing catastrophic risks. Our paper examines what these evaluations can and cannot tell us. Evaluations can establish lower bounds on AI capabilities and assess certain misuse risks given sufficient effort from evaluators.

Unfortunately, evaluations face fundamental limitations that cannot be overcome within the current paradigm. These include an inability to establish upper bounds on capabilities, reliably forecast future model capabilities, or robustly assess risks from autonomous AI systems. This means that while evaluations are valuable tools, we should not rely on them as our main way of ensuring AI systems are safe. We conclude with recommendations for incremental improvements to frontier AI safety, while acknowledging these fundamental limitations remain unsolved.

[Read more](https://arxiv.org/abs/2412.08653)

![](https://intelligence.org/wp-content/uploads/2024/12/https_intelligence.org_wp-content_uploads_2024_11_Mechanisms-to-Verify-International-Agreements-About-AI-Development-27-Nov-24_page-0001-1.jpg)

## Mechanisms to Verify International Agreements About AI Development

International agreements about AI development may be required to reduce catastrophic risks from advanced AI systems. However, agreements about such a high-stakes technology must be backed by verification mechanisms—processes or tools that give one party greater confidence that another is following the agreed-upon rules, typically by detecting violations. This report gives an overview of potential verification approaches for three example policy goals, aiming to demonstrate how countries could practically verif

... (truncated, 8 KB total)
Resource ID: c3e44c68ce4e78a5 | Stable ID: NGIxZDAxNj