Analysis of a Monte Carlo Tree Search in Knight-Amazons

Abstract

Abstract We study whether or not a Monte Carlo tree search program can find a winning strategy in a combinatorial game. For this purpose we propose a new combinatorial game Knight-Amazons which is a variant of the game of Amazons. Two players start the game Knight-Amazons by placing several black and white knights on the specified cells on a board. The first player selects and moves one of the white knights, and then, he or she chooses any empty cell in the range of the knight moved and thwarts it. No piece cannot be placed on the thwarted thereafter. Similarly, the second player selects and moves one of the black knights, and chooses any empty cell in the range of the knight moved and thwarts it. Two players play alternately and then the player who can no longer move any knight and thwart a cell becomes a loser of the game. And we prove that the second player at Knight-Amazons of n × n board, where n is even, can always win if he or she follows Tweedledum-Tweedledee strategy. Since there are multiple winning ways for a second player in Knight-Amazons of n × n board, where n is even, Knight-Amazons is a desired platform to study behavior of MCTS. We are interested in whether or not a MCTS program can find any winning way. We analyze the game tree of Knight-Amazons and compute win-loss ratio in several game states in a 4 × 4 board. Comparing the obtained win-loss ratios, we examine behavior of Monte Carlo tree search (MCTS) in Knight-Amazons. Then we execute an upper confidence bounds applied to trees (UCT) program as MCTS and find which moves the UCT program chooses most often. The result indicates that the UCT program does not necessarily converge to a Tweedledum-Tweedledee strategy nor moves having high win-loss ratio even when the number of playouts increases. It follows that a basic MCTS program cannot find a winning way such as Tweedledum-Tweedledee strategy.