Emergent bluffing and inference with Monte Carlo Tree Search

Abstract

In many card and board games, players cannot see the whole game state, with different players seeing different parts of the state. In such games, gathering of information (inference) is a key strategic aspect, to which information hiding (bluffing, among other techniques) is an important countermeasure. Monte Carlo Tree Search (MCTS) is a powerful general-purpose technique for decision making in games. MCTS rose to prominence through successes in combinatorial board games such as Go, but more recently has demonstrated promise in card, board and video games of incomplete information. MCTS can construct robust plans in stochastic environments (making it strong in some games), but in its vanilla form is unable to infer or bluff (making it weak in games where this is a central feature). In this paper, we augment MCTS with mechanisms for performing inference and bluffing. Like all algorithms based on game tree search, MCTS implicitly constructs a model of the opponents' decision processes. We show that this model can be repurposed to perform an approximation of Bayesian inference. We also obtain bluffing behaviour by self-determinization (introducing “impossible” worlds into the agent's pool of sampled states). We test our algorithms on The Resistance, a popular card game based around hidden roles.