Exploring the first-move balance point of Go-Moku based on reinforcement learning and Monte Carlo tree search

Abstract

In most chess games without additional rule restrictions, the side that makes the first move (i.e., the first-move side) has an absolute advantage, which affects the game’s balance to a certain extent. Artificial intelligence (AI) training in some chess games can be bottlenecked by the imbalance of opening moves, making it challenging to improve chess strength. The first-move balance problem can be explored to achieve a balanced win rate in chess games. This study uses Go-Moku as an example to explore the first-move balance point problem for different sizes of Go-Moku boards. We design a self-playing Go-Moku intelligence algorithm using deep reinforcement learning and Monte Carlo tree search (MCTS), which can considerably save arithmetic power without affecting the strength of the AI. To address the characteristics of Go-Moku and its complexity, we propose an algorithm using dynamic MCTS simulation counts, which only employs a reasonable amount of hyperparameters to achieve better performance with the cost of a relatively small number of simulations. By symmetrically expanding the data and optimizing the exploration and selection allocation, the training efficiency of the Go-Moku AI is improved through Multiple Process Interface (MPI) multi-processes. Building the test model of first-move balance points for a universal Go-Moku board, we obtain a set of first-move balance points for different board sizes. The first-move balance point of Go-Moku that makes the game even is found by simulating the game win rate for all first-move drop points. The experimental results demonstrate that the proposed algorithm can achieve world-leading chess strength in Gomocup by engine play tests and can find the first-move balance point of Go-Moku on boards of various sizes. The results of this study will help optimize the rule setting of Go-Moku and improve the training efficiency of AI in the field of Go-Moku, which can be extended to the exploration of balance in other chess games.