OpenAI - How We Think About Safety Alignment

How we think about safety and alignment | OpenAI OpenAI How we think about safety and alignment

 The mission of OpenAI is to ensure artificial general intelligence (AGI) benefits all of humanity. Safety—the practice of enabling AI’s positive impacts by mitigating the negative ones—is thus core to our mission. 

 Our understanding of how to advance safety has evolved a lot over time, and this post is a current snapshot of the principles that guide our thinking. We are not certain everything we believe is correct. We do know AI will transform most aspects of our world, and so we should think through the benefits, changes, and risks of this technology early. 

 AGI in many steps rather than one giant leap

 We used to view the development of AGI as a discontinuous moment when our AI systems would transform from solving toy problems to world-changing ones. We now view the first AGI as just one point along a series of systems of increasing usefulness. 

 In a discontinuous world, practicing for the AGI moment is the only thing we can do, and safety lessons come from treating the systems of today with outsized caution relative to their apparent power. This is the approach we took for GPT‑2 when we initially didn’t release the model due to concerns about malicious applications ⁠ . 

 In the continuous world, the way to make the next system safe and beneficial is to learn from the current system. This is why we’ve adopted the principle of iterative deployment ⁠ , so that we can enrich our understanding of safety and misuse, give society time to adapt to changes, and put the benefits of AI into people’s hands. At present, we are navigating the new paradigm of chain-of-thought models ⁠ - we believe this technology will be extremely impactful going forward, and we want to study how to make it useful and safe by learning from its real-world usage. In the continuous world view, deployment aids rather than opposes safety. 

 These diverging views of the world lead to different interpretations of what is safe. For example, our release of ChatGPT was a Rorschach test for many in the field—depending on whether they expected AI progress to be discontinuous or continuous, they viewed it as either a detriment or learning opportunity towards AGI safety. 

 Impacts of AGI

 We are developing AGI because we believe in its potential to positively transform everyone’s lives. Almost any challenge facing humanity feels surmountable with a sufficiently capable AGI because intelligence has been responsible for most improvements for humanity, from literacy to machines to medicine. 

 Nevertheless, intelligence is a neutral term, and intelligence alone is not a guarantee for positive transformation. Achieving AGI’s potential includes working diligently to mitigate the potential harms of increasingly powerful AI systems, and developing and operating them in accordance with human values and with humans in control. 

 As AI becomes more powerful, the stakes grow higher. The exac

... (truncated, 20 KB total)