Sagan (1993) "Limits of Safety"

web

press.princeton.edu·press.princeton.edu/books/paperback/9780691021010/the-lim...

A foundational text in catastrophic risk studies frequently cited in AI safety discussions as an analogy for how well-intentioned safety systems can fail in complex high-stakes domains; the organizational failure modes Sagan identifies are directly applicable to AI deployment risks.

Metadata

Importance: 72/100bookprimary source

Summary

Scott Sagan's seminal 1993 book examines near-accidents and safety failures in U.S. nuclear weapons operations, arguing that complex military organizations are structurally prone to accidents despite best efforts. Drawing on Normal Accident Theory and organizational theory, Sagan challenges the optimistic 'High Reliability Organization' view and demonstrates that catastrophic nuclear accidents have been narrowly avoided multiple times through luck rather than design.

Key Points

•Applies Normal Accident Theory to nuclear weapons, arguing complex tightly-coupled systems make serious accidents nearly inevitable over time.
•Documents multiple declassified near-accidents with U.S. nuclear forces during the Cold War, revealing how close civilization came to catastrophe.
•Challenges High Reliability Organization (HRO) theory, arguing military bureaucracies have structural incentives that undermine safety culture.
•Demonstrates that luck, not just design, has prevented nuclear catastrophe—raising serious concerns about long-term safety of nuclear arsenals.
•Highly relevant to AI safety as a case study in how complex high-stakes systems can fail despite institutional safeguards and safety intentions.

Cited by 1 page

Page	Type	Quality
Autonomous Weapons Escalation Model	Analysis	62.0

Cached Content Preview

HTTP 200Fetched Mar 15, 20268 KB

The Limits of Safety | Princeton University Press 
 
 
 

 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 

 
 

 
 
 

 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 The Limits of Safety

 Scott Douglas Sagan 

 
 
 
 
 
 Paperback 

 ISBN: 9780691021010 
 
 
 
 $58.00/£48.00 
 
 
 
 
 
 
 ebook (PDF via app) 

 ISBN: 9780691213064 
 
 
 
 $58.00/£48.00 
 
 
 

 
 
 
 

 
 Shipping to: 
 
 
 
 
 
 Choose Country 
 United States 
 Canada 
 United Kingdom 
 Afghanistan 
 Aland Islands 
 Albania 
 Algeria 
 American Samoa 
 Andorra 
 Angola 
 Anguilla 
 Antarctica 
 Antigua And Barbuda 
 Argentina 
 Armenia 
 Aruba 
 Australia 
 Austria 
 Azerbaijan 
 Bahamas 
 Bahrain 
 Bangladesh 
 Barbados 
 Belarus 
 Belgium 
 Belize 
 Benin 
 Bermuda 
 Bhutan 
 Bolivia 
 Bonaire, Sint Eustatius and Saba 
 Bosnia And Herzegovina 
 Botswana 
 Bouvet Island 
 Brazil 
 British Indian Ocean Territory 
 Brunei Darussalam 
 Bulgaria 
 Burkina Faso 
 Burundi 
 Cabo Verde 
 Cambodia 
 Cameroon 
 Cayman Islands 
 Central African Republic 
 Chad 
 Chile 
 China 
 Christmas Island 
 Cocos (Keeling) Islands 
 Colombia 
 Comoros 
 Congo 
 Congo, Democratic Republic 
 Cook Islands 
 Costa Rica 
 Cote D'Ivoire 
 Croatia 
 Cuba 
 Curaçao 
 Cyprus 
 Czech Republic 
 Denmark 
 Djibouti 
 Dominica 
 Dominican Republic 
 Ecuador 
 Egypt 
 El Salvador 
 Equatorial Guinea 
 Eritrea 
 Estonia 
 Ethiopia 
 Falkland Islands (Malvinas) 
 Faroe Islands 
 Fiji 
 Finland 
 France 
 French Guiana 
 French Polynesia 
 French Southern Territories 
 Gabon 
 Gambia 
 Georgia 
 Germany 
 Ghana 
 Gibraltar 
 Greece 
 Greenland 
 Grenada 
 Guadeloupe 
 Guam 
 Guatemala 
 Guernsey 
 Guinea 
 Guinea-Bissau 
 Guyana 
 Haiti 
 Heard Island & Mcdonald Islands 
 Holy See (Vatican City State) 
 Honduras 
 Hong Kong 
 Hungary 
 Iceland 
 India 
 Indonesia 
 Iran, Islamic Republic Of 
 Iraq 
 Ireland 
 Isle Of Man 
 Israel 
 Italy 
 Jamaica 
 Japan 
 Jersey 
 Jordan 
 Kazakhstan 
 Kenya 
 Kiribati 
 Korea 
 Korea People' Republic Of 
 Kuwait 
 Kyrgyzstan 
 Lao People's Democratic Republic 
 Latvia 
 Lebanon 
 Lesotho 
 Liberia 
 Libyan Arab Jamahiriya 
 Liechtenstein 
 Lithuania 
 Luxembourg 
 Macao 
 Macedonia 
 Madagascar 
 Malawi 
 Malaysia 
 Maldives 
 Mali 
 Malta 
 Marshall Islands 
 Martinique 
 Mauritania 
 Mauritius 
 Mayotte 
 Mexico 
 Micronesia, Federated States Of 
 Moldova 
 Monaco 
 Mongolia 
 Montenegro 
 Montserrat 
 Morocco 
 Mozambique 
 Myanmar 
 Namibia 
 Nauru 
 Nepal 
 Netherlands 
 New Caledonia 
 New Zealand 
 Nicaragua 
 Niger 
 Nigeria 
 Niue 
 Norfolk Island 
 Northern Mariana Islands 
 Norway 
 Oman 
 Pakistan 
 Palau 
 Palestinian Territory, Occupied 
 Panama 
 Papua New Guinea 
 Paraguay 
 Peru 
 Philippines 
 Pitcairn 
 Poland 
 Portugal 
 Puerto Rico 
 Qatar 
 Reunion 
 Romania 
 Russian Federation 
 Rwanda 
 Saint Barthelemy 
 Saint Helena 
 Saint Kitts And Nevis 
 Saint Lucia 
 Saint Martin 
 Saint Pierre And Miquelon

... (truncated, 8 KB total)

Resource ID: 1c3d683813eb5f58 | Stable ID: YjE4M2ZkZj