Future of Humanity Institute (FHI)

Quick Assessment

Organization Details

Overview

The Future of Humanity Institute (FHI) was a multidisciplinary research center at the University of Oxford that fundamentally shaped how humanity thinks about long-term risks and the future of civilization. Founded by philosopher Nick Bostrom in November 2005 as part of the Oxford Martin School (then the James Martin 21st Century School), FHI brought together researchers from philosophy, computer science, mathematics, and economics to tackle questions that most of academia considered too speculative or far-fetched to study rigorously.

During its 19-year existence, FHI achieved an extraordinary record of intellectual impact relative to its modest size. The institute was involved in the germination of a wide range of ideas that have since become mainstream concerns: existential risk, effective altruism, longtermism, AI alignmentApproachAI AlignmentComprehensive review of AI alignment approaches finding current methods (RLHF, Constitutional AI) achieve 75-90% effectiveness on existing systems but face critical scalability challenges, with ove...Quality: 91/100, AI governanceParameterAI GovernanceThis page contains only component imports with no actual content - it displays dynamically loaded data from an external source that cannot be evaluated., global catastrophic risk, information hazards, the unilateralist's curse, and moral uncertainty. Starting with just three researchers in 2005, FHI grew to approximately fifty at its peak before administrative conflicts with Oxford's Faculty of Philosophy led to a hiring freeze in 2020 and ultimate closure in April 2024.

FHI's influence extends far beyond its publications. The institute trained a generation of researchers who now hold leadership positions at AnthropicOrganizationAnthropicComprehensive profile of Anthropic, founded in 2021 by seven former OpenAI researchers (Dario and Daniela Amodei, Chris Olah, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish) with early funding..., DeepMind, OpenAIOrganizationOpenAIComprehensive organizational profile of OpenAI documenting evolution from 2015 non-profit to commercial AGI developer, with detailed analysis of governance crisis, safety researcher exodus (75% of ..., the Centre for the Governance of AIOrganizationGovAIGovAI is an AI policy research organization with ~15-20 staff, funded primarily by Coefficient Giving ($1.8M+ in 2023-2024), that has trained 100+ governance researchers through fellowships and cur...Quality: 43/100 (GovAI), and numerous other organizations. Toby OrdPersonToby OrdComprehensive biographical profile of Toby Ord documenting his 10% AI extinction estimate and role founding effective altruism, with detailed tables on risk assessments, academic background, and in...Quality: 41/100's The Precipice was quoted by UK Prime Minister Boris Johnson in his 2021 UN General Assembly address, and FHI researchers advised the UN Secretary General's Office on existential risk and future generations. The institute's closure represents the end of an era, but its intellectual legacy continues through its alumni, spin-off organizations, and the fields it created.

Historical Evolution

Founding Era (2005-2008)

Nick Bostrom established FHI in November 2005 after recognizing that questions about humanity's long-term future and existential risks were being systematically neglected by mainstream academia. The institute was initially funded as a three-year pilot project but quickly demonstrated its value through a series of influential publications and conferences.

In its early years, FHI focused on establishing existential risk as a legitimate field of academic inquiry. Bostrom's 2002 paper "Existential Risk: Analyzing Human Extinction Scenarios" had laid the conceptual groundwork, defining existential risk as "one where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential." FHI's task was to build an institutional home for this research.

Growth Period (2008-2014)

Between 2008 and 2010, FHI hosted the Global Catastrophic Risks conference, wrote 22 academic journal articles, and published 34 chapters in academic volumes. The GCR conference was a pivotal moment in building an academic community around reducing risks to humanity's future.

This period saw FHI expand its research scope significantly. When Bostrom began work on a book about existential risk in 2009, he found one chapter on AI "getting out of hand." The issue of risks from superintelligent systems turned out to be much deeper than initially expected, eventually evolving into Superintelligence: Paths, Dangers, Strategies, published in 2014.

Peak Period (2014-2020)

The publication of Superintelligence marked the beginning of FHI's most influential period. The book became an international bestseller and is credited with convincing many technologists, including Elon Musk and Bill Gates, to take AI risks seriously. FHI grew to approximately 50 researchers and received its largest funding commitments.

In 2018, FHI received a series of awards totaling up to £13.3 million over three years from Coefficient GivingOrganizationCoefficient GivingCoefficient Giving (formerly Open Philanthropy) has directed $4B+ in grants since 2014, including $336M to AI safety (~60% of external funding). The organization spent ~$50M on AI safety in 2024, w...Quality: 55/100 (then Open Philanthropy), the largest donation in the history of the Faculty of Philosophy at Oxford. This funding supported work on risks from advanced AI, biosecurity and pandemic preparedness, and macrostrategy.

Decline and Closure (2020-2024)

Despite its intellectual success, FHI's final years were marked by what Anders Sandberg called "gradual suffocation by Faculty bureaucracy." The flexible, fast-moving approach of the institute did not function well with the rigid rules and slow decision-making of the surrounding organization.

The stated reason for closure, according to Bostrom, was that the university did not have the operational bandwidth to manage FHI. Sandberg explained the cultural mismatch: "I often described Oxford like a coral reef of calcified institutions built on top of each other... FHI was one such fish but grew too big for its hole. At that point it became either vulnerable to predators, or had to enlarge the hole, upsetting the neighbors."

Research Programs and Contributions

Existential Risk Studies

FHI essentially created the academic field of existential risk studies. Before FHI, the topic was considered too speculative for serious academic attention. FHI demonstrated that it was possible to do rigorous research on big-picture questions about humanity's future.

The 2018 paper "Dissolving the Fermi Paradox" by Sandberg, Drexler, and Ord was the first to estimate and rigorously account for uncertainties in each term of the Drake equation. They found a high likelihood that we are alone in our galaxy or even the entire observable universe.

AI Safety Research

FHI was one of the earliest academic institutions to take AI safety seriously, working in close collaboration with labs such as DeepMind, OpenAI, and CHAI (Center for Human-Compatible AI).

Stuart Armstrong's collaboration with DeepMind on "Interruptibility" was mentioned in over 100 media articles and represented one of FHI's more practical AI safety contributions.

AI Governance

FHI's Governance of AI (GovAI) team, led by Allan Dafoe, became the largest research group focused on the policy implications of advanced AI before spinning out as an independent organization in 2021.

GovAI spun out of Oxford in 2021 specifically to "escape bureaucratic restrictions" and has since become an independent nonprofit. Allan Dafoe now heads DeepMind's Long-Term AI Strategy and Governance Team.

Macrostrategy Research

FHI's Macrostrategy group examined how long-term outcomes for humanity are connected to present-day actions—a research program that influenced the effective altruism movement's focus on cause prioritization.

Biosecurity Research

FHI's Biosecurity group worked on making the world more secure against both natural and human-made catastrophic biological risks, anticipating many concerns that became urgent during the COVID-19 pandemic.

Key Personnel

Nick Bostrom (Founder and Director)

Bostrom is best known for his work in five areas: existential risk, the simulation argument, anthropics, impacts of future technology, and implications of consequentialism for global strategy. His simulation argument posits that one of three propositions must be true: civilizations almost never reach technological maturity, technologically mature civilizations are uninterested in running simulations, or we are almost certainly living in a simulation.

Toby Ord (Senior Research Fellow)

Ord's The Precipice was the first book-length treatment of existential risk for a wide audience, influencing policy in the United Kingdom and at the United Nations. Multiple FHI staff were invited to present their work to the British Parliament.

Anders Sandberg (Senior Research Fellow)

Sandberg authored the FHI Final Report, which provides a detailed account of the institute's history and achievements. He is described as a futurist who explored the outer limits of what advanced civilizations might achieve.

Stuart Armstrong (Research Fellow)

Armstrong's research centered on how to define AI goals and map humanity's partially-defined values into AI systems. His collaboration with DeepMind on interruptibility was one of FHI's most visible practical contributions.

Eric Drexler (Senior Research Fellow)

Drexler, best known for pioneering the concept of molecular nanotechnology, brought a unique perspective to AI safety. His "Reframing Superintelligence" proposed an alternative to the "single superintelligent agent" model that dominated much AI safety thinking.

Carl Shulman (Research Associate)

Shulman contributed to FHI's work on forecasting AI development timelines and understanding the strategic implications of advanced AI.

Other Notable Researchers

Major Publications

Books

Influential Papers

Funding and Resources

Coefficient Giving Grants

Total Coefficient Giving funding exceeded $10 million over FHI's lifetime.

Other Major Funders

Budget and Operations

FHI's annual revenues and expenses were approximately £1 million per year at operational scale, with the bulk of funding from academic grants that were "lumpy and hard to predict."

Policy Impact

United Nations

United Kingdom

European Union

FHI researchers contributed to policy discussions on AI governance that informed the development of the EU AI Act.

Spin-offs and Related Organizations

Direct Spin-offs

Related Organizations at Oxford

Organizations Influenced by FHI

Reasons for Closure

Official Reasons

The University stated it did not have "operational bandwidth" to manage FHI. The institute cited "increasing administrative headwinds within the Faculty of Philosophy."

Detailed Analysis

Anders Sandberg's Explanation

In the FHI Final Report, Sandberg explained:

"While FHI had achieved significant academic and policy impact, the final years were affected by a gradual suffocation by Faculty bureaucracy. The flexible, fast-moving approach of the institute did not function well with the rigid rules and slow decision-making of the surrounding organization."

He used the metaphor of Oxford as a "coral reef of calcified institutions built on top of each other," with FHI as a fish that grew too big for its hole.

Contextual Factors

The university explicitly stated that the email controversy was not a factor in the closure decision, according to Bostrom.

Legacy and Assessment

Intellectual Legacy

Institutional Legacy

FHI demonstrated that it was possible to do rigorous academic research on big-picture questions about humanity's future. Topics that once "struggled to eke out a precarious existence at the margins of a single philosophy department are now pursued by leading AI labs, government agencies, nonprofits, and specialized academic research centers."

Alumni Impact

Assessment of Impact

Lessons and Implications

For Research Institutes

For the AI Safety Field

FHI's closure coincides with AI safety becoming mainstream. As Bostrom noted, "There is now a much broader support base for the kind of work FHI was set up to enable, so the institute essentially served its purpose." The question is whether the distributed ecosystem of organizations can match FHI's record of fundamental advances.

Open Questions

• Did FHI's administrative troubles reflect fixable problems or inherent tensions between academic institutions and existential risk research?
• Will the distributed ecosystem of FHI successor organizations be as productive as the concentrated institute?
• What institutional models best support long-term, speculative research?

Sources and Citations

Primary Sources

• Future of Humanity Institute Website (archived)
• FHI New Website
• FHI Final Report (Sandberg, 2024)
• Nick Bostrom's Homepage

Wikipedia and Reference Sources

• Future of Humanity Institute - Wikipedia
• Nick Bostrom - Wikipedia
• Anders Sandberg - Wikipedia
• Superintelligence: Paths, Dangers, Strategies - Wikipedia

News Coverage

• Nature: Future of Humanity Institute shuts
• Daily Nous: The End of the Future of Humanity Institute
• Oxford Student: Oxford shuts down Elon Musk-funded FHI
• Asterisk Magazine: Looking Back at the Future of Humanity Institute

Grant Records

• Coefficient Giving: FHI General Support
• Coefficient Giving: FHI Work on Global Catastrophic Risks
• Oxford University: £13.3m boost for FHI

Community Discussion

• EA Forum: Future of Humanity Institute
• EA Forum: FHI Final Report Discussion
• LessWrong: FHI has shut down