The hierarchical task network planning method based on Monte Carlo Tree Search

Abstract

Since the hierarchical task network (HTN) planning depends on the domain knowledge of the problem, the planning result relies on the writing order of the decomposition method. Besides, the solution obtained by planning is usually a general feasible solution, which means there are shortcomings in the ability of finding the optimal solution. In order to reduce the dependence of HTN planning on domain knowledge and obtain a better planning solution, Pyhop-m, an HTN planning algorithm based on Monte Carlo Tree Search(MCTS) is proposed. In the planning process, a planning tree is built by MCTS to guide the HTN planner to choose the best decomposition method. Experiments illustrates that whether in the static or dynamic environment, Pyhop-m is superior to the existing Pyhop and heuristic-based Pyhop-h in plan length, planning success rate and optimal solution rate. Under the 95% confidence level, the confidence intervals of Pyhop-m algorithm to achieve the planning success rate and the optimal solution rate in the dynamic environment are [75.82%,89.18%] and [88.67%,93.95%], which are significantly higher than those of Pyhop-h with [58.19%,77.81%] and [69.91%,80.69%], respectively. Moreover, it can solve the planning problem with uncertain action executions by repeatedly simulating and evaluating the leaf nodes of the planning tree. It can be concluded that Pyhop-m can not only make the planning result independent of the writing order of the decomposition methods, but also search out the global optimal solution.