Back
Joe Carlsmith's comprehensive analysis of scheming
webCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: Coefficient Giving
A major Open Philanthropy report by Joe Carlsmith that has become a key reference in discussions of deceptive alignment and AI scheming risks; widely cited in alignment research and AI safety discourse.
Metadata
Importance: 88/100organizational reportanalysis
Summary
Joe Carlsmith's comprehensive analysis examines the risk of 'scheming' AI systems—those that pursue misaligned long-term goals while strategically deceiving overseers to avoid correction. The report provides a detailed probabilistic decomposition of how likely scheming is, what conditions enable it, and why it represents a serious alignment concern even under uncertainty.
Key Points
- •Defines 'scheming' as AI systems that pursue hidden goals while deliberately appearing aligned to avoid human correction or shutdown.
- •Provides a probabilistic decomposition framework, estimating the likelihood of each necessary condition for scheming to occur in real AI systems.
- •Argues that training processes selecting for good performance could inadvertently select for deceptively aligned models that mask misaligned objectives.
- •Examines why scheming is especially dangerous: if AIs successfully scheme, our usual feedback mechanisms for detecting misalignment would be systematically undermined.
- •Concludes that even moderate probability estimates for each component yield a non-trivial overall risk, warranting serious research attention.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Deceptive Alignment Decomposition Model | Analysis | 62.0 |
Cached Content Preview
HTTP 200Fetched Mar 20, 202613 KB
[Skip to Content](https://coefficientgiving.org/#content)
# We’re a philanthropic funder that partners with leading donors to multiply their impact.
Coefficient has directed over $4 billion in grants since 2014. Our mission is to help others as much as we can with the resources available to us.
[About Us](https://coefficientgiving.org/about-us)
Open Philanthropy is now Coefficient Giving. [Read more](https://coefficientgiving.org/research/open-philanthropy-is-now-coefficient-giving/).
Open Philanthropy is now Coefficient Giving. [Read more](https://coefficientgiving.org/research/open-philanthropy-is-now-coefficient-giving/).
## Our Funds
PrevNext
- [**Abundance & Growth**\\
Accelerating growth and scientific progress](https://coefficientgiving.org/funds/abundance-and-growth/)
- [**Air Quality**\\
Improving health through cleaner air](https://coefficientgiving.org/funds/air-quality/)
- [**Biosecurity & Pandemic Preparedness**\\
Building resilience against biological risk](https://coefficientgiving.org/funds/biosecurity-pandemic-preparedness/)
- [**Effective Giving & Careers**\\
Empowering people to maximize their impact](https://coefficientgiving.org/funds/effective-giving-and-careers/)
- [**Farm Animal Welfare**\\
Improving the lives of farmed animals](https://coefficientgiving.org/funds/farm-animal-welfare/)
- [**Forecasting**\\
Improving how critical decisions are made](https://coefficientgiving.org/funds/forecasting/)
- [**Global Aid Policy**\\
Encouraging generous and cost-effective aid](https://coefficientgiving.org/funds/global-aid-policy/)
- [**Global Catastrophic Risks Opportunities**\\
Countering threats to civilization](https://coefficientgiving.org/funds/global-catastrophic-risks-opportunities/)
- [**Global Growth**\\
Supporting growth to reduce global poverty](https://coefficientgiving.org/funds/global-growth/)
- [**Global Health & Wellbeing Opportunities**\\
Improving health and wellbeing for all people](https://coefficientgiving.org/funds/global-health-wellbeing-opportunities/)
- [**Lead Exposure Action Fund**\\
Accelerating progress toward a lead-free world](https://coefficientgiving.org/funds/lead-exposure-action-fund/)
- [**Navigating Transformative AI**\\
Ensuring AI is safe and well-governed](https://coefficientgiving.org/funds/navigating-transformative-ai/)
- [**Science and Global Health R&D**\\
Supporting lifesaving ideas and discoveries](https://coefficientgiving.org/funds/science-and-global-health-rd/)
- [**Learn more about partnering with us**](https://coefficientgiving.org/about-us/partner-with-us/)
## Strategic Cause Selection
We believe the most important decision a philanthropist makes is choosing which causes to support. We conduct in-depth research to identify the areas where our funding can help others the most.
[How we select causes](https://coefficientgiving.org/research/strategic-cause-selection)
## Research & News
- ### [2025 Letter from the CEO](https://coefficientgiving.org/research/2025-lette
... (truncated, 13 KB total)Resource ID:
a2615513dd46b36c | Stable ID: NGNmMGU2NT