Mastering the game of Go with deep neural networks and tree search
paperAuthors
Lars Chittka·Vera Vasas
Credibility Rating
5/5
Gold(5)Gold standard. Rigorous peer review, high editorial standards, and strong institutional reputation.
Rating inherited from publication venue: Nature
AlphaGo is a foundational capabilities milestone often cited in AI safety discussions about rapid capability jumps, forecasting AI progress, and the implications of superhuman AI performance in complex domains.
Metadata
Importance: 72/100journal articleprimary source
Summary
This landmark Nature paper presents AlphaGo, DeepMind's system that defeated professional human Go players by combining deep convolutional neural networks with Monte Carlo tree search. It demonstrated that deep learning could master a domain previously thought to require human-like intuition, representing a major AI capabilities milestone. The work showed that value networks and policy networks trained on human expert data and self-play could achieve superhuman performance in a complex strategic game.
Key Points
- •AlphaGo combines policy networks (to select moves) and value networks (to evaluate positions) trained via supervised learning on expert games and reinforcement learning via self-play.
- •First AI system to defeat a professional human Go player without handicap, previously considered a decade away from achievement at the time.
- •Uses Monte Carlo Tree Search guided by neural networks to reduce the search space in Go's astronomically large game tree (~10^170 positions).
- •Demonstrated that deep RL and self-play could achieve superhuman performance in a domain requiring long-horizon strategic planning and intuition.
- •Represents a pivotal capabilities milestone with implications for AI progress forecasting and the trajectory toward general problem-solving systems.
1 FactBase fact citing this source
Cached Content Preview
HTTP 200Fetched May 11, 202622 KB
Subjects
Computational science
Computer science
Reward
Abstract
The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.
Access through your institution
Buy or subscribe
This is a preview of subscription content, access via your institution
Access options
Access through your institution
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Learn more
Buy this article
Purchase on SpringerLink
Instant access to the full article PDF.
USD 39.95
Prices may be subject to local taxes which are calculated during checkout
Figure 1: Neural network training pipeline and architecture. The alternative text for this image may have been generated using AI. Figure 2: Strength and accuracy of policy and value networks. The alternative text for this image may have been generated using AI. Figure 3: Monte Carlo tree search in AlphaGo. The alternative text for this image may have been generated using AI. Figure 4: Tournament evaluation of AlphaGo. The alternative text for this image may have been generated using AI. Figure 5: How AlphaGo (black, to play) selected its move in an informal game against Fan Hui. The alternative text for this image may have been generated using AI. Figure 6: Games from the match between AlphaGo and the European champion, Fan Hui. The alternative text for this image may have been generated using AI.
Similar content being viewed by others
Construction, observation and knowledge abstraction for go endgames on small boards
Article
Open access
22 March 2024
The neural basis of intuitive approximate number system in board game Go (Baduk) expert
... (truncated, 22 KB total)Resource ID:
0df4be45b473dda2 | Stable ID: sid_1fY31i5QVu