ImS-MCTS: A New Importance Sampling MCTS Algorithm for Discrete Event System

Abstract

With the emergence of a discrete event system (DES), the research community is developing in the direction of various issues within it. In this kind of system, the decision-making problem is one of the core issues. In this paper, we build a new decision-making algorithm ImS-MCTS based on Monte Carlo Tree Search (MCTS). As for decision-making problems, MCTS family is a state-of-the-art kind of algorithms, this kind of algorithm can be decomposed into tree search and Monte Carlo (MC) method. The MC method approximates the value function of the current state by simulating future states. Researchers face the challenge of capturing rare events during simulation to make accurate estimations, resulting in a huge number of samples being required, which is often impractical in reality. To solve this problem, we propose ImS-MCTS to improve the traditional MC method through importance sampling. The improvement reduces the sampling frequency of non-rare events by probability mapping, effectively reduces the sample space, and helps tree search compute more efficiently. We design experiments in autonomous driving. The results show that: compared with traditional MCTS methods, ImS-MCTS can improve the success rate of collision avoidance by about 7.12% and reduce the time cost by about 6.24% in the test road section, which indicates that ImS-MCTS makes better decisions.