AlphaZero shows a good simulator now beats a store of expert data

One self-taught algorithm mastered chess, shogi and Go in a day. Where rules can be simulated, compute replaces expert data and the moat moves to the simulator.
Twenty-eight wins, 72 draws and no losses: that is how AlphaZero, a program from Alphabet's DeepMind given nothing but the rules, finished a 100-game chess match against Stockfish, the 2016 computer chess champion. The paper, posted on 5 December, describes the same algorithm reaching superhuman play in chess, shogi and Go within 24 hours of training, beating the champion shogi program 90 games to 8 and the three-day version of AlphaGo Zero 60 to 40.
The result changes what counts as a moat in machine learning. Where a problem can be simulated exactly, a system that plays against itself no longer needs human examples or hand-built expertise, and the scarce inputs become the simulator and the computing power to run it. Data advantages built on years of expert records lose their value in any domain where a machine can generate better records itself.
Stockfish embodies years of hand-tuned rules from open-source developers. AlphaZero passed it after four hours of training, with judgment rather than brute force: it evaluates about 80,000 chess positions per second against Stockfish's 70 million, and its network picks the few lines that deserve attention. In October the AlphaGo Zero paper had shown the same principle in Go alone, beating the version that defeated Lee Sedol 100 games to 0 after three days of self-play with no human games at all. Two months later the method carries over to two more games without changes.
The cost sits in a specific place. Training used 5,000 first-generation TPUs, Google's own chips for machine learning, to generate self-play games and 64 second-generation TPUs to train the networks. The finished player then ran on a single machine with four TPUs. Learning is expensive and running the result is cheap, the same split that new phone chips exploit, which favors whoever can pay for one large training run and sell the outcome many times. Few companies outside Alphabet can fund that today.
Founders should read it as a map of where to stand. A company that owns an accurate simulator of a costly problem, from scheduling a fleet to balancing a power grid, holds the asset that self-play needs, and the computing bill can be rented. A company whose edge is a store of expert decisions in a domain that can be simulated holds an asset that is losing value. Fanaura puts one question early to founders who bring us a machine learning company: can a machine with enough compute regenerate its data advantage?
Board games are the objection. They have exact rules and no hidden information, and business has neither. Critics of the chess match question its one-minute-per-move time control and Stockfish's settings, and the work is still a preprint. The test is a game with hidden information and real-time play: DeepMind and Blizzard opened StarCraft II as a research environment in August, and a strong self-play result there would show the method leaving the board.
Until then the lesson is narrow but sharp. Wherever the rules can be written down, expertise can now be computed, and what we back has to rest on something that cannot.
Building an AI-native company? Apply in five minutes. →
More from Fanaura
Building a company that owns a simulator of a costly real-world problem? Send us the project.
Fanaura invests its own capital and the capital of its limited partners (LPs), usually through a separate SPV for each deal. If we see a fit, we get in touch.
Get the news by email
One short note each weekday. No marketing, and you can unsubscribe at any time.