Governance.ai: Open Problems in Technical AI Governance
governmentCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: Centre for the Governance of AI
Published by Governance.ai, this paper is a key reference for researchers seeking to understand where technical AI work can most meaningfully support real-world AI governance and policy implementation.
Metadata
Importance: 78/100organizational reportanalysis
Summary
This paper defines and scopes 'technical AI governance' (TAIG) as the application of technical analysis and tools to support effective AI governance. It presents a structured taxonomy of open problems organized around three functions: identifying where governance intervention is needed, assessing the efficacy of governance actions, and designing enforcement and compliance mechanisms. The work is intended to guide technical researchers and funders toward high-impact contributions at the intersection of AI policy and technical research.
Key Points
- •Introduces 'technical AI governance' (TAIG) as a distinct field bridging technical AI research and policy/governance needs.
- •Organizes open problems into three core functions: identifying intervention needs, assessing governance efficacy, and designing enforcement/compliance mechanisms.
- •Highlights a critical gap: decision-makers lack sufficient technical information to identify risks and evaluate governance options.
- •Serves as a prioritization resource for researchers and funders seeking to direct effort toward policy-relevant technical problems.
- •Covers topics including model evaluation, monitoring, auditing, compute governance, and standards development.
Cached Content Preview
HTTP 200Fetched May 17, 202698 KB
Open Problems in Technical AI Governance
Anka Reuel\* Stanford University [anka.reuel@stanford.edu](mailto:anka.reuel@stanford.edu)
Ben Bucknall\* Centre for the Governance of AI & [ben.bucknall@govrance.ai](mailto:ben.bucknall@govrance.ai)
Oxford Martin AI Governance Initiative
Stephen Casper MIT CSAIL
Tim Fist Institute for Progress & Center for a New American Security
Lisa Soder interface - Tech Analysis and Policy Ideas for Europe e.V.
Omni Aarne Institute for AI Policy and Strategy
Lewis Hammond University of Oxford & Cooperative AI Foundation
Lajibin Brabhin University of Oxford
Alan Chan Centre for the Governance of AI & Mila
Peter Wills Centre for the Governance of AI & University of Oxford
Markus Anderljung Centre for the Governance of AI
Ben Garfinkel Centre for the Governance of AI
Lennart Heim Centre for the Governance of AI
Andrew Trask OpenMined & University of Oxford
Gabriel Mukobi Standard University
Rylan Schaeffer Stanford University
Mauricio Baker Independent Researcher
Sara Hooker Colere For AI
Irene Solaiman Hugging Face
Alexandra Sasha Luccioni Hugging Face
Nitarshan Rajkumar University of Cambridge
Nicola Mosk The Future Society
Neel Guha Stanford University
Jessica Newman University of California, Berkeley
Yoshua Bengio University of Montreal & Mila
Tobin South MIT
Alex Pentland Stanford HAI
Jeffrey Ladish Palaiside Research
Samui Koyogi Stanford University
Mykel J. Kochenderfer Stanford University
Robert Trager Oxford Martin AI Governance Initiative, Blauutik School of Government &
University of Oxford
Abstract
AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.
\*Equal contribution; corresponding authors; order randomized.
Given its scope, inclusion as an author does not entail endorsement of all aspects of the paper, with the exception of AR and BB.
Cite as Reuel, Bucknall, et al. (2024) “Open Problems in Technical AI Governance.”
* * *
1 Introduction
The rapid development and adoption of artificial intelligence (AI) systems¹ has prompted a great deal of governance action from the public sector², academia and civil society (Anderljung et al., 2023a; Moes & Ryan, 2023; Barrett et al., 2023), and industry (Anthropic, 2023a; Microsof
... (truncated, 98 KB total)Resource ID:
b4402422d8628b71 | Stable ID: sid_48VhCMzfuz