Meta AI's Self-Rewarding Language Models

The Unavoidable Problem of Self-Improvement in AI: An Interview with Ramana Kumar, Part 1 - Future of Life Institute 
 Skip to content The Unavoidable Problem of Self-Improvement in AI: An Interview with Ramana Kumar, Part 1 

 Published: March 19, 2019 Author: Jolene Creighton Contents 

 Today’s AI systems may seem like intellectual powerhouses that are able to defeat their human counterparts at a wide variety of tasks. However, the intellectual capacity of today’s most advanced AI agents is, in truth, narrow and limited . Take, for example, AlphaGo. Although it may be the world champion of the board game Go , this is essentially the only task that the system excels at. 

 Of course, there’s also AlphaZero . This algorithm has mastered a host of different games, from Japanese and American chess to Go. Consequently, it is far more capable and dynamic than many contemporary AI agents; however, AlphaZero doesn’t have the ability to easily apply its intelligence to any problem. It can’t move unfettered from one task to another the way that a human can. 

 The same thing can be said about all other current AI systems — their cognitive abilities are limited and don’t extend far beyond the specific task they were created for. That’s why Artificial General Intelligence (AGI) is the long-term goal of many researchers. 

 Widely regarded as the “holy grail” of AI research , AGI systems are artificially intelligent agents that have a broad range of problem-solving capabilities, allowing them to tackle challenges that weren’t considered during their design phase. Unlike traditional AI systems, which focus on one specific skill, AGI systems would be able efficiently to tackle virtually any problem that they encounter, completing a wide range of tasks. 

 If the technology is ever realized, it could benefit humanity in innumerable ways. Marshall Burke, an economist at Stanford University, predicts that AGI systems would ultimately be able to create large-scale coordination mechanisms to help alleviate (and perhaps even eradicate) some of our most pressing problems, such as hunger and poverty. However, before society can reap the benefits of these AGI systems, Ramana Kumar , an AGI safety researcher at DeepMind , notes that AI designers will eventually need to address the self-improvement problem. 

 Self-Improvement Meets AGI

 Early forms of self-improvement already exist in current AI systems. “There is a kind of self-improvement that happens during normal machine learning,” Kumar explains; “namely, the system improves in its ability to perform a task or suite of tasks well during its training process.” 

 However, Kumar asserts that he would distinguish this form of machine learning from true self-improvement because the system can’t fundamentally change its own design to become something new. In order for a dramatic improvement to occur — one that encompasses new skills, tools, or the creation of more advanced AI agents — current AI systems need a human 

... (truncated, 10 KB total)