Human-level Control Through Deep Reinforcement Learning (Nature DQN Paper)
paperCredibility Rating
Gold standard. Rigorous peer review, high editorial standards, and strong institutional reputation.
Rating inherited from publication venue: Nature
A foundational deep RL capabilities paper from DeepMind; while not directly about AI safety, it is essential background for understanding modern RL-based agents and is frequently cited in reward learning, specification, and alignment research.
Metadata
Summary
This landmark 2015 Nature paper introduces Deep Q-Networks (DQN), combining Q-learning with deep convolutional neural networks to learn control policies directly from raw pixel inputs. DQN achieves human-level performance across 49 Atari 2600 games using a single architecture and hyperparameter set, enabled by two key innovations: experience replay and a separate target network for stable training. It represents a foundational breakthrough in deep reinforcement learning, demonstrating that a single agent can master diverse complex tasks end-to-end.
Key Points
- •Introduces DQN, which combines Q-learning with deep CNNs to learn directly from high-dimensional raw pixel inputs without hand-crafted features.
- •Achieves human-level or better performance on 49 Atari 2600 games using one algorithm, network architecture, and set of hyperparameters.
- •Key technical innovations: experience replay buffer (breaks data correlations) and a frozen target network (stabilizes training).
- •Draws inspiration from neuroscience, noting parallels between temporal-difference learning and dopaminergic neural signaling in biological agents.
- •Foundational capabilities paper that accelerated deep RL research and influenced subsequent alignment-relevant work on reward learning and agent evaluation.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Google DeepMind | Organization | 37.0 |
Cached Content Preview
Human-level control through deep reinforcement learning | Nature
Skip to main content
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain
the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in
Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles
and JavaScript.
Advertisement
Subjects
Computer science
Abstract
The theory of reinforcement learning provides a normative account 1 , deeply rooted in psychological 2 and neuroscientific 3 perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems 4 , 5 , the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms 3 . While reinforcement learning agents have achieved some successes in a variety of domains 6 , 7 , 8 , their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks 9 , 10 , 11 to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games 12 . We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in th
... (truncated, 22 KB total)456ab451e9b31397 | Stable ID: OTBkYzgwMj