What does AI’s success playing complex board games tell brain scientists?

Abstract

In research, it sometimes happens that advances in one field unexpectedly inform another in a fundamental way. A case in point may be what computer-generated gameplay suggests about how brains operate. Spearheaded by Demis Hassabis, David Silver, and their colleagues at the artificial intelligence (AI) company Google DeepMind, a series of articles published over the past several years has reported ongoing improvement of a computational strategy (1⇓–3) that now beats all players in complex board games (4). Beginning in 2015 with a computer program that beat human opponents at all 49 games in the Atari 2600 suite (challenges that include Pong, Space Invaders, and Pac-Man) (1), the group progressed to a program called AlphaGo that beat the European and world champions at Go (2, 3), a territorial board game far more complicated than chess. The latest version in this effort, called AlphaZero (4), now beats the best players—human or machine—in chess and shogi (Japanese chess) as well as Go. Fig. 1. The search space of the territorial board game Go is intractable using logical algorithms. Image credit: Shutterstock/Saran Poroong. That machines can beat expert human players in board games is not news. The achievement of this long-standing goal in computer science first attracted widespread attention in 1997 when IBM's program “Deep Blue” beat Garry Kasparov, then the world champion chess player (5). Human players have since been shown to be weak opponents in such games compared with a variety of machine programs. What is different about Google DeepMind’s AlphaZero is that the system operates by learning on a wholly empirical (trial and error) basis. Previous game-playing machines, including earlier versions in the AlphaGo series, used a combination of brute force logic, tree search, successful moves made by experts in response to specific board positions, and other “hand-crafted” expedients. … [↵][1]1Email: purves{at}neuro.duke.edu. [1]: #xref-corresp-1-1