All news

OpenAI Five's Dota win makes game worlds a machine learning asset

A tabletop diorama of a game world with a river and two stone fortresses, light trails along its paths

OpenAI's bots beat the Dota 2 world champions after 45,000 years of self-play. Game worlds are now the cheapest place to train machine learning at scale.

The world champions of Dota 2 lost twice in a row to software on 13 April. OpenAI Five, a team of five neural networks trained by playing against copies of itself, beat OG 2-0 in San Francisco, the first AI system to defeat world champions at an esports game.

The result says less about Dota than about where machine learning now gets trained. OpenAI expected to need new algorithmic ideas and found that the problem mostly needed scale: about 45,000 years of self-play over ten months, up from roughly 10,000 years over a month and a half when the same system lost both its matches at The International last August. Experience on that scale is affordable only in a game or simulation. That turns rich game worlds into an input machine learning cannot do without, and it hands game companies a technology they can use in their own products.

The bill is large. The Finals version had consumed 800 petaflop/s-days of compute, a measure of sustained processing, and it played with a pool of 17 heroes rather than the full roster. OpenAI says the power of current reinforcement learning comes at the cost of massive amounts of experience, which can be impractical outside a game or simulated environment, and calls reducing that experience the next challenge. It has retired OpenAI Five as a competitor, and the same training system has already taught a robotic hand to turn a block, trained entirely in simulation.

What matters most for game companies arrived as a surprise. Trained only to beat copies of itself, the system played competently alongside humans without extra training, and at the Finals it filled three seats on each side of a mixed match, with two human players per team. From 18 to 21 April OpenAI opened it to the public in an event called Arena, and it won 7,215 of 7,257 competitive games, or 99.4%, and played tens of thousands more games as a teammate. A bot that can carry a novice, fill an empty seat in a team game or play a new patch across thousands of matches before release is a product, and studios already pay people for the playtesting and balancing it would take over. Such tools sit within the game studios and tools Fanaura invests in.

Cost is the weak point of that case. A system that needed ten months of training and a restricted hero pool for a single game is a research result, not a feature a mid-sized studio can buy, and game rules change with every patch. OpenAI has kept training the same system since June 2018 through game patches and changes to the model without starting over. The deciding signal is the technical analysis OpenAI promised after Arena, and above all whether the experience needed to reach this level starts to fall. Until it does, agents of this class stay in labs with large compute budgets, and the opportunity for studios and tool makers is to be ready when the price drops. Fanaura wants to meet the teams preparing for that moment, the founders we work with from the first build.

For founders

Building AI teammates, playtesting agents or simulation tools for games? Send us the project.

Fanaura invests its own capital and the capital of its limited partners (LPs), usually through a separate SPV for each deal. If we see a fit, we get in touch.

Apply in five minutes
Subscribe

Get the news by email

One short note each weekday. No marketing, and you can unsubscribe at any time.