Research on AI Safety (Russell et al., AIPS 2015)

Summary

This paper by Stuart Russell and colleagues, presented at AIPS 2015, outlines a foundational framework for AI safety centered on the idea that AI systems should be uncertain about human values and use inverse reinforcement learning to infer them. It introduces the concept of assistance games (formerly CIRL) where AI agents are cooperative and defer to human preferences rather than pursuing fixed objective functions.

Key Points

• •Argues that specifying a fixed objective function for AI is dangerous; instead, AI should remain uncertain about human values and seek to learn them.
• •Proposes inverse reinforcement learning (IRL) as a mechanism for AI to infer human preferences from observed behavior.
• •Introduces cooperative/assistance game framework where AI and human have aligned incentives, making the AI inherently deferential.
• •Identifies three core properties for safe AI: altruistic (human-welfare-maximizing), uncertainty about objectives, and ability to learn from human behavior.
• •Frames AI safety as a technical problem solvable through principled probabilistic and game-theoretic approaches.