AI for AI for Epistemics

# AI for AI for Epistemics
By owencb, Lukas Finnveden
Published: 2026-04-01
We feel conscious that rapid AI progress could transform all sorts of cause areas. But we haven’t previously analysed what this means for AI for epistemics, a field close to our hearts. In this article, we attempt to rectify this oversight.

Summary
-------

AI-powered tools and services that help people figure out what’s true (“AI for epistemics”) could matter a lot. 

As R&D is increasingly automated, AI systems will play a larger role in the process of developing such AI-based epistemic tools. This has important implications. Whoever is willing to devote sufficient compute will be able to build strong versions of the tools, quickly. Eventually, the hard part won’t be building useful systems, but making sure people trust the right ones, and making sure that they are truth-tracking even in domains where that’s hard to verify.

We can do some things now to prepare. Incumbency effects mean that shaping the early versions for the better could have persistent benefits. Helping build appetite among socially motivated actors with deep pockets could enable the benefits to come online sooner, and in safer hands. And in some cases, we can identify particular things that seem likely to be bottlenecks later, and work on those directly.

Background: AI for epistemics
-----------------------------

AI for epistemics — i.e. getting AI systems to give more truth-conducive answers, and building tools that help the epistemics of the users — seems like a big deal to us. Some past things we’ve written on the topic include:

*   [Truthful AI](https://arxiv.org/abs/2110.06674)
*   [What’s Important in “AI for Epistemics”?](https://www.forethought.org/research/whats-important-in-ai-for-epistemics)
*   [AI Tools for Existential Security](https://www.forethought.org/research/ai-tools-for-existential-security)
*   [Design sketches: collective epistemics](https://www.forethought.org/research/design-sketches-collective-epistemics)
*   [Design sketches: tools for strategic awareness](https://www.forethought.org/research/design-sketches-tools-for-strategic-awareness)

These past articles mostly take the perspective of “how can people build AI systems which do better by these lights?”. But maybe we should be thinking much more about what changes when people can use AI tools to do increasingly large fractions of the development work!

The shift in what drives AI-for-epistemics progress
---------------------------------------------------

Right now, AI-for-epistemics tools are constrained by two main bottlenecks: the quality of the underlying AI systems, and whether people have invested serious development effort in building the tools to use those systems.

The balance of bottlenecks is changing. Two years ago, the quality of underlying AI systems was the central bottleneck. Today, it is much less so — many useful tools could probably work based on current LLMs. It is likely still a constraint on how g

... (truncated, 18 KB total)