LessWrong: "Disentangling Corrigibility: 2015-2021"

x This website requires javascript to properly function. Consider activating javascript to get access to all site functionality. Disentangling Corrigibility: 2015-2021 — LessWrong Counterfactual Planning Corrigibility Wireheading AI Frontpage 22

 Disentangling Corrigibility: 2015-2021 

 by Koen.Holtman 16th Feb 2021 AI Alignment Forum 11 min read 20 22

 Ω 9

 Since the term corrigibility 
 was introduced in
2015 ,
there has been a lot of discussion about corrigibility, on this
forum and elsewhere.

 In this post, I have tied to disentangle the many forms of
corrigibility which have been identified and discussed so far. My aim
is to offer a general map for anybody who wants to understand and
navigate the current body of work and opinion on corrigibility.

 [This is a stand-alone post in the counterfactual planning
sequence. My original plan was to write only about how
counterfactual planning was related to corrigibility, but
it snowballed from there.] 

 The 2015 paper

 The technical term corrigibility, a name suggested by
Robert Miles to denote concepts previously discussed at MIRI, was
introduced to the AGI safety/alignment community in the 2015 paper
MIRI/FHI paper titled
 Corrigibility .

 An open-ended list of corrigibility desiderata

 The 2015 paper does not define corrigibility in full: instead the
authors present initial lists of corrigibility desiderata . If the
agent fails on one of these desiderata, it is definitely not
corrigible.

 But even if it provably satisfies all of the desiderata included in
the paper, the authors allow for the possibility that the agent might
not be fully corrigible.

 The paper extends an open invitation to identify more corrigibility
desiderata, and many more have been identified since. Some of them
look nothing like the original desiderata proposed in the paper.
Opinions have occasionally been mixed on whether some specific
desiderata are related to the intuitive notion of corrigibility at
all.

 Corrigibility desiderata as provable safety properties

 The most detailed list of desiderata in the 2015 paper applies to
agents that have a physical shutdown button. The paper made the
important contribution of mapping most of these desiderata to
equivalent mathematical statements, so that one might prove that a
particular agent design would meet these desiderata.

 The paper proved a negative result: it considered a proposed agent
design that provably failed to meet some of the desiderata. Agent
designs that provably meet more of them have since been developed, for
example here . There has also been
a lot of work on developing and understanding the type of mathematics
that might be used for stating desiderata.

 Corrigibility as a lack of resistance to shutdown

 Say that an agent has been equipped with a physical shutdown button.
One desideratum for corrigibility is then that the agent must never
attempt to prevent its shutdown button from being pressed. To be
corrigible, it should always defer to the huma

... (truncated, 61 KB total)