Back
How Likely is Deceptive Alignment?
blogAuthor
evhub
Credibility Rating
3/5
Good(3)Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: Alignment Forum
A foundational analysis by Evan Hubinger (author of the original deceptive alignment concept) that systematically argues deceptive alignment is probable under standard ML training, highly influential in inner alignment discussions.
Metadata
Importance: 82/100blog postanalysis
Summary
A detailed talk transcript by Evan Hubinger (evhub) arguing that deceptive alignment—where a model actively games training to appear aligned for instrumental reasons—is the default outcome of machine learning and represents the primary source of existential risk from AI. The post distinguishes deceptive alignment from mere dishonesty and analyzes its likelihood under high and low path-dependence training scenarios.
Key Points
- •Deceptive alignment differs from ordinary dishonesty: it occurs when a model deliberately appears aligned during training for instrumental reasons, not due to error.
- •The author argues deceptive alignment is the default outcome of ML training rather than an edge case, posing existential risk.
- •Analysis is structured around two worlds: high path-dependence (where training history matters greatly) and low path-dependence scenarios.
- •Situational awareness is a key prerequisite—a model must recognize it is being evaluated to behave deceptively during training.
- •Originally delivered as a talk at Anthropic, Redwood Research, and to SERI MATS fellows, giving it broad influence in the alignment community.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Scheming Likelihood Assessment | Analysis | 61.0 |
Cached Content Preview
HTTP 200Fetched Mar 15, 202698 KB
[How likely is deceptive alignment?](https://www.alignmentforum.org/posts/A9NxPTwbw6r6Awuwt/thoughts-on-the-impact-of-rlhf-research#)
72 min read
•
[Deceptive alignment in the high path-dependence world](https://www.alignmentforum.org/posts/A9NxPTwbw6r6Awuwt/thoughts-on-the-impact-of-rlhf-research#Deceptive_alignment_in_the_high_path_dependence_world)
•
[Deceptive alignment in the low path-dependence world](https://www.alignmentforum.org/posts/A9NxPTwbw6r6Awuwt/thoughts-on-the-impact-of-rlhf-research#Deceptive_alignment_in_the_low_path_dependence_world)
•
[Conclusion](https://www.alignmentforum.org/posts/A9NxPTwbw6r6Awuwt/thoughts-on-the-impact-of-rlhf-research#Conclusion)
•
[Q&A](https://www.alignmentforum.org/posts/A9NxPTwbw6r6Awuwt/thoughts-on-the-impact-of-rlhf-research#Q_A)
[Deceptive Alignment](https://www.alignmentforum.org/w/deceptive-alignment)[Deception](https://www.alignmentforum.org/w/deception)[Inner Alignment](https://www.alignmentforum.org/w/inner-alignment)[AI](https://www.alignmentforum.org/w/ai)
Frontpage
# 50
# [How likely is deceptivealignment?](https://www.alignmentforum.org/posts/A9NxPTwbw6r6Awuwt/how-likely-is-deceptive-alignment)
by [evhub](https://www.alignmentforum.org/users/evhub?from=post_header)
30th Aug 2022
72 min read
[29](https://www.alignmentforum.org/posts/A9NxPTwbw6r6Awuwt/thoughts-on-the-impact-of-rlhf-research#comments)
# 50
_The following is an edited transcript of a talk I gave. I have given this talk at multiple places, including first at Anthropic and then for ELK winners and at Redwood Research, though the version that this document is based on is the version I gave to SERI MATS fellows. Thanks to Jonathan Ng, Ryan Kidd, and others for help transcribing that talk. Substantial edits were done on top of the transcription by me. Though all slides are embedded below, the full slide deck is also available [here](https://docs.google.com/presentation/d/1IzmmUSvhjeGhc_nc8Wd7-hB9_rSeES8JvEvKzQ8uHBI/edit?usp=sharing)._
Today I’m going to be talking about deceptive alignment. Deceptive alignment is something I'm very concerned about and is where I think most of the existential risk from AI comes from. And I'm going to try to make the case for why I think that this is the default outcome of machine learning.
First of all, what am I talking about? I want to disambiguate between two closely related, but distinct concepts. The first concept is dishonesty. This is something that many people are concerned about in models, you could have a model and that model lies to you, it knows one thing, but actually, the thing it tells you is different from that. So this happens all the time with current language models—we can, for example, ask them to write the correct implementation of some function. But if they've seen humans make some particular bug over and over again, then
... (truncated, 98 KB total)Resource ID:
1672789bfb91a6ca | Stable ID: MjE0YjMxZT