Modification of improved upper confidence bounds for regulating exploration in Monte-Carlo tree search

Abstract

The key to the success of the upper confidence bounds on trees (UCT) algorithm, which is the most widely adopted variant of the Monte-Carlo tree search (MCTS) algorithm, is that it combines MCTS with the upper confidence bounds (UCB) bandit algorithm. The improved UCB algorithm is a bandit algorithm that has a superior regret bound to the UCB algorithm. However, some characteristics of the improved UCB algorithm are not suitable for direct application to MCTS. The combined confidence bounds (CCB) bandit algorithm is a modification of the improved UCB algorithm, making it more suitable for the task of tree searches. The CCB bandit algorithm can be further extended to regulate exploration in simple regret minimization in MCTS, through the exploration regulating factor.In this paper, we present an analysis of the bound on simple regret for the CCB bandit algorithm. We also give a comprehensive overview of how different choices of the exploration factor impact the games of 9×9 Go and 9×9 Nogo.