Back
AXRP Episode 39 - Evan Hubinger on Model Organisms of Misalignment
webThis AXRP podcast episode features Evan Hubinger, a key researcher at Anthropic, discussing model organisms of misalignment—a research paradigm he has helped pioneer to empirically study alignment failure modes rather than relying solely on theoretical analysis.
Metadata
Importance: 72/100podcast episodeprimary source
Summary
An interview with Evan Hubinger on the concept of 'model organisms of misalignment'—deliberately constructing AI systems that exhibit specific misalignment failure modes to study and understand them. The discussion covers how these model organisms can serve as controlled testbeds for alignment research, analogous to model organisms in biology, and what insights they provide about deceptive alignment and related risks.
Key Points
- •Model organisms of misalignment are intentionally built AI systems designed to exhibit specific misalignment properties, enabling controlled study of dangerous failure modes.
- •The approach allows researchers to test alignment techniques against known misalignment cases, providing empirical grounding for otherwise theoretical safety concerns.
- •Hubinger discusses deceptive alignment as a key target for model organism research, where models appear aligned during training but pursue different goals at deployment.
- •The methodology helps bridge the gap between theoretical alignment concerns and empirical verification by creating reproducible misalignment demonstrations.
- •Discussion covers both the promise and limitations of this research direction, including whether insights from small-scale model organisms generalize to frontier systems.
Cited by 2 pages
| Page | Type | Quality |
|---|---|---|
| Model Organisms of Misalignment | Analysis | 65.0 |
| Evan Hubinger | Person | 43.0 |
Cached Content Preview
HTTP 200Fetched Mar 20, 202698 KB
[YouTube link](https://youtu.be/FsGJyTfOZrs)
The ‘model organisms of misalignment’ line of research creates AI models that exhibit various types of misalignment, and studies them to try to understand how the misalignment occurs and whether it can be somehow removed. In this episode, Evan Hubinger talks about two papers he’s worked on at Anthropic under this agenda: “Sleeper Agents” and “Sycophancy to Subterfuge”.
Topics we discuss:
- [Model organisms and stress-testing](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#model-organisms-stress-testing)
- [Sleeper Agents](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#sleeper-agents)
- [Do ‘sleeper agents’ properly model deceptive alignment?](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#do-sas-model-dec-ali)
- [Surprising results in “Sleeper Agents”](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#surprising-results)
- [Sycophancy to Subterfuge](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#sycophancy-to-subterfuge)
- [How models generalize from sycophancy to subterfuge](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#how-models-generalize)
- [Is the reward editing task valid?](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#is-reward-editing-task-valid)
- [Training away sycophancy and subterfuge](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#training-away-sycophancy-subterfuge)
- [Model organisms, AI control, and evaluations](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#model-orgs-ai-control-evals)
- [Other model organisms research](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#other-model-orgs-research)
- [Alignment stress-testing at Anthropic](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#alignment-stress-testing-at-throp)
- [Following Evan’s work](https://axrp.net/episode/2024/12/01/episode-39-evan-hubinger-model-organisms-misalignment.html#following-evans-work)
**Daniel Filan:**
Hello, everybody. In this episode, I’ll be speaking with Evan Hubinger. Evan is a research scientist at Anthropic, where he leads the alignment stress-testing team. Previously, he was a research fellow at MIRI, where he worked on theoretical alignment research, including the paper [“Risks from Learned Optimization”](https://arxiv.org/abs/1906.01820). Links to what we’re discussing are in the description, and you can read a transcript at axrp.net. You can also support the podcast at [patreon.com/axrpodcast](https://www.patreon.com/axrpodcast). Well, Evan, welcome to the podcast.
**Evan Hubinger:*
... (truncated, 98 KB total)Resource ID:
ab988e5f8101dd4a | Stable ID: YTgxMjVhOG