Determinization and information set Monte Carlo Tree Search for the card game Dou Di Zhu

Abstract

Determinization is a technique for making decisions in games with stochasticity and/or imperfect information by sampling instances of the equivalent deterministic game of perfect information. Monte-Carlo Tree Search (MCTS) is an AI technique that has recently proved successful in the domain of deterministic games of perfect information. This paper studies the strengths and weaknesses of determinization coupled with MCTS on a game of imperfect information, the popular Chinese card game Dou Di Zhu. We compare a “cheating” agent (with access to hidden information) to an agent using determinization with random deals. We investigate the fraction of knowledge that a non-cheating agent could possibly infer about opponents' hidden cards. Furthermore, we show that an important source of error in determinization arises since this approach searches a tree that does not truly resemble the game tree for a game with stochasticity and imperfect information. Hence we introduce a novel variant of MCTS that operates directly on trees of information sets and show that our algorithm performs well in precisely those situations where determinization using random deals performs poorly.