Experiments on Learning Unit-Action Models from Replay Data from RTS Games

Abstract

Recent work has shown that incorporating action probability models (models that given a game state can predict the probability with which an expert will play each move) into MCTS can lead to significant performance improvements in a variety of adversarial games, including RTS games. This paper presents a collection of experiments aimed at understanding the relation between the amount of training data, the predictive performance of the action models, the effect of these models in the branching factor of the game and the resulting performance gains in MCTS. Experiments are carried out in the context of the microRTS simulator, showing that more accurate predictive models do not necessarily result in better MCTS performance.