Guiding Monte Carlo Tree Search by Scripts in Real-Time Strategy Games

Abstract

In Real-Time Strategy (RTS) games, the action space grows combinatorially with respect to the number of units. With limited computing budget between actions, methods like Monte Carlo Tree Search (MCTS) tend to get lost in the massive search space. An interesting line of existing work is to incorporate human knowledge in the form of scripts. In this paper, we investigate different possibilities for incorporating scripts into the tree policy while still maintaining the convergence guarantees of MCTS. We also report experiments on incorporating the scripts into the playout policy, which showed that unbiased bots perform better than biased bots.