Science of Deep Learning More Tractably Addresses the Sharp Left Turn Than Agent Foundations

x This website requires javascript to properly function. Consider activating javascript to get access to all site functionality. Science of Deep Learning more tractably addresses the Sharp Left Turn than Agent Foundations — LessWrong AI Frontpage 22

 Science of Deep Learning more tractably addresses the Sharp Left Turn than Agent Foundations 

 by NickGabs 19th Sep 2023 7 min read 2 22

 Summary

 Lots of agent foundations research is motivated by the idea that alignment techniques found by empirical trial-and-error will fail to generalize to future systems. While such threat models are plausible, agent foundations researchers have largely failed to make progress on addressing them because they tend to make very few assumptions about the gears-level properties of future AI systems. In contrast, the emerging field of "Science of Deep Learning" (SDL) assumes that the systems under question are neural networks, and aims to empirically uncover general principles about how they work. Assuming such principles exist and that neural nets will lead to AGI, insights from SDL will generalize to future systems even if specific alignment techniques found by trial-and-error do not. As a result, SDL can address the threat models that motivate agent foundations research while being much more tractable due to making more assumptions about future AI systems, though it cannot help in the construction of an ideal utility function for future AI systems to maximize.

 Thanks to Alexandra Bates for discussion and feedback 

 Introduction

 Generally speaking, the "agent foundations" line of research in alignment is motivated by a particular class of threat models: that alignment techniques generated through empirical trial and error like RLHF or Constitutional AI will suddenly break down as AI capabilities advance, leading to catastrophe. For example, Nate Soares has argued that AI systems will undergo a   "sharp left turn" at which point the "shallow" motivational heuristics which caused them to be aligned at lower capabilities levels will cease to function. Eliezer Yudkowsky seems to endorse a similar threat model in   AGI Ruin , arguing that rapid capabilities increases will cause models to go out of distribution and become misaligned.

 I'm much more optimistic about finding robust alignment methods through trial and error than these researchers, but I'm not here to debate that. While I think sharp left turn style failure modes are relatively unlikely, they do not seem totally implausible. Moreover, because doom is much more likely in worlds where alignment techniques generated through trial and error fail to generalize, it makes sense to   focus on them when doing alignment research. Instead, I want to argue that the emerging field of the "science of deep learning" or SDL is more promising for dealing with these threat models than agent foundations.

 A Central Tradeoff in Alignment Research

 AI alignment research agendas face a central tradeoff: robus

... (truncated, 12 KB total)