Monte-Carlo Tree Search with Neural Networks for Petri Nets

Abstract

This paper considers the tracking control of Petri nets, namely finding the optimal firing sequence that leads the Petri net from an initial marking to a destination marking. Value neural networks (VNN) and policy neural networks (PNN) are used to improve the Monte-Carlo Tree Search (MCTS) based tracking control approach proposed recently in [1]. It is shown how to integrate the VNN and PNN, respectively, with the simulation and expansion step of the MCTS algorithm, so that the search space is significantly reduced. By introducing the neural networks, the dependence of the performance of the MCTS algorithm on parameter selection is also strongly reduced. Compared with the existing tracking control approaches, the proposed approaches can handle large PNs and have a very high probability of finding the optimal firing sequence within a prespecified time. The PNN based MCTS approach needs less online calculation, while the VNN based MCTS approach requires less offline training time. An example is given to illustrate the proposed approaches and show the advantage of the proposed approaches over other approaches.