Back
OpenAI Five | OpenAI
webCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: OpenAI
OpenAI Five is a key capabilities milestone illustrating what large-scale RL can achieve; relevant to AI safety discussions about capability jumps, emergent behavior, and the compute-performance relationship.
Metadata
Importance: 55/100blog postprimary source
Summary
OpenAI Five was a reinforcement learning system that achieved superhuman performance in Dota 2, a complex real-time strategy game, by training using self-play at massive scale. It demonstrated that large-scale RL with sufficient compute could master long-horizon, multi-agent cooperative and competitive tasks previously considered intractable. The project served as a landmark capabilities demonstration and provided insights into emergent teamwork, strategy, and scaling.
Key Points
- •Achieved superhuman Dota 2 performance, defeating world champions OG at OpenAI Five Finals in April 2019.
- •Trained via large-scale self-play reinforcement learning, accumulating roughly 45,000 years of gameplay experience.
- •Demonstrated emergent cooperative behaviors among five AI agents without explicit coordination mechanisms programmed.
- •Highlighted the role of compute scaling in enabling AI to tackle complex, long-horizon multi-agent environments.
- •Provided lessons relevant to AI safety regarding reward shaping, multi-agent dynamics, and policy robustness.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Deep Learning Revolution Era | Historical | 44.0 |
Cached Content Preview
HTTP 200Fetched Feb 22, 202621 KB
OpenAI Five | OpenAI Switch to ChatGPT (opens in a new window)
Sora (opens in a new window)
API Platform (opens in a new window)
OpenAI June 25, 2018
Milestone OpenAI Five
Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2.
Loading… Share Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2 (opens in a new window) . While today we play with restrictions , we aim to beat a team of top professionals at The International (opens in a new window) in August subject only to a limited set of heroes. We may not succeed: Dota 2 is one of the most popular and complex (opens in a new window) esports games in the world, with creative and motivated professionals who train (opens in a new window) year-round to earn part of Dota’s annual $40M prize pool (opens in a new window) (the largest of any esports game).
OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of Proximal Policy Optimization running on 256 GPUs and 128,000 CPU cores—a larger-scale version of the system we built to play the much-simpler solo variant of the game last year. Using a separate LSTM (opens in a new window) for each hero and no human data, it learns recognizable strategies. This indicates that reinforcement learning (opens in a new window) can yield long-term planning with large but achievable scale—without fundamental advances, contrary to our own expectations upon starting the project.
To benchmark our progress, we’ll host a match versus top players on August 5th. Follow (opens in a new window) us on Twitch to view the live broadcast, or request (opens in a new window) an invite to attend in person!
The problem
One AI milestone is to exceed human capabilities in a complex video game like StarCraft (opens in a new window) or Dota. Relative to previous AI milestones like Chess (opens in a new window) or Go (opens in a new window) , complex video games start to capture the messiness and continuous nature of the real world. The hope is that systems which solve complex video games will be highly general, with applications outside of games.
Dota 2 is a real-time strategy game played between two teams of five players, with each player controlling a character called a “hero”. A Dota-playing AI must master the following:
Long time horizons. Dota games run at 30 frames per second for an average of 45 minutes, resulting in 80,000 ticks per game. Most actions (like ordering a hero to move (opens in a new window) to a location) have minor impact individually, but some individual actions like town portal (opens in a new window) usage can affect the game strategically; some strategies (opens in a new window) can play out over an entire game. OpenAI Five observes every fourth frame, yielding 20,000 moves. Chess (opens in a new
... (truncated, 21 KB total)Resource ID:
e388403add7d489b | Stable ID: MmZhZmQ5YT