Back
Claire Berlinski Substack - Is the AI Control Problem Insoluble?
blogCredibility Rating
2/5
Mixed(2)Mixed quality. Some useful content but inconsistent editorial standards. Claims should be verified.
Rating inherited from publication venue: Substack
A journalistic interview format making Yampolskiy's formal insolubility thesis accessible to general audiences; useful for understanding pessimistic perspectives on AI controllability, though not a peer-reviewed technical treatment.
Metadata
Importance: 48/100interviewcommentary
Summary
Claire Berlinski interviews AI safety researcher Roman Yampolskiy, who argues that the AI control problem is formally and inherently insoluble—we can improve safety incrementally but cannot achieve the 100% safety needed to prevent existential risk from superintelligent systems. The piece balances Yampolskiy's pessimistic technical thesis with Berlinski's concern that defeatism could undermine safety research efforts.
Key Points
- •Yampolskiy argues the AI control problem is not merely difficult but formally insoluble, meaning no solution can guarantee complete safety.
- •Incremental safety improvements are possible, but 100% safety—necessary to prevent existential catastrophe—is argued to be unachievable in principle.
- •Berlinski expresses skepticism but treats Yampolskiy's credentials seriously, framing the conclusion as genuinely alarming rather than dismissible.
- •The piece cautions against defeatism: acknowledging insolubility should not halt safety research but should increase urgency before superintelligence is developed.
- •The interview highlights tension between formal impossibility arguments and practical safety work in the AI alignment community.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| ControlAI | Organization | 63.0 |
Cached Content Preview
HTTP 200Fetched Mar 15, 202611 KB
Is the AI control problem insoluble?
Subscribe Sign in The Cosmopolicast Is the AI control problem insoluble? 13 14 2 1× 0:00 Current time: 0:00 / Total time: -47:20 -47:20 Audio playback is not supported on your browser. Please upgrade. Is the AI control problem insoluble?
A conversation with Roman Yampolskiy Jun 04, 2023 13 14 2 Share Transcript
“ … unfortunately we show that the AI control problem is not solvable and the best we can hope for is safer AI , but ultimately not 100 percent safe AI, which is not a sufficient level of safety in the domain of existential risk as it pertains to humanity.”
Roman Yampolskiy is a computer scientist and professor at the Speed School of Engineering at the University of Louisville who works on genetic algorithms, neural networks, artificial intelligence, and the alignment problem.
Our conversation surprised me for two reasons. First, he’s the only researcher to whom I’ve spoken who argues that GPT4 is conscious. Second—much more gravely—he believes that we’ll not only fail to solve the control problem before we build a dangerously intelligent AI, but that the problem is inherently and formally insoluble.
As I’ve reflected on control problem this week, I’ve had the growing and uneasy suspicion that this must be so. That said, had you told me five years ago what Large Language Models would be doing in 2023, I would have said that was impossible, too. My intuitions about which problems in AI engineering are soluble aren’t trustworthy. It takes years of working on problems like these to develop good intuitions, and I haven’t done that.
That can’t be said of Roman, however. This is his life’s work. We have every good reason to take his intuitions seriously. So when I heard him say that, my heart sank. He may be wrong, he tried to reassure me. He hopes he is. I hope he is. But he doesn’t think he is.
Our conversation was perfectly calm, as you’ll hear, but that’s because I just can’t bring myself to believe in any of this, despite the evidence. This exchange and its implications seem no more real to me than a thought experiment in a graduate philosophy seminar or a science fiction movie. That I feel this way shows that awareness of one’s cognitive biases is no proof against them. In reality, it’s neither a thought experiment nor a movie; it’s perfectly plausible that he’s right, and if so, we’re in indescribably big trouble.
As hard as it is to take this in, we have to, because this hasn’t happened yet. It may be difficult to stop it at this point, but at least it’s not formally impossible. Once it happens? Too late.
So it’s worth thinking
... (truncated, 11 KB total)Resource ID:
8bbb2dcdb898692e | Stable ID: ZjBkYzE3Y2