Population based Monte Carlo tree search hyper-heuristic for combinatorial optimization problems

Abstract

Hyper-heuristics aim to automate the heuristic selection process in order to operate well across different problem instances, or even across different problem domains. A traditional hyper-heuristic framework has two levels, a high level strategy and a set of low level heuristics. The role of the high level strategy is to decide which low level heuristic should be executed at the current decision point. This paper proposes a Monte Carlo tree search hyper-heuristic framework. We model the search space of the low level heuristics as a tree and use Monte Carlo tree search to search through the tree in order to identify the best sequence of low level heuristics to be applied to the current state. To improve the effectiveness of the proposed framework, we couple it with a memory mechanism which contains a population of solutions, utilizing different population updating rules. The generality of the proposed framework is demonstrated using the six domains of the hyper-heuristic competition (CHeSC) test suite (boolean satisfiability (MAX-SAT), one dimensional bin packing, permutation flow shop, personnel scheduling, traveling salesman and vehicle routing with time windows). The results demonstrate that the proposed hyper-heuristic generalizes well over all six domains and obtains competitive, if not better results, when compared to the best known results that have previously been presented in the scientific literature.