A novel reinforcement learning-based hyper-heuristic for heterogeneous vehicle routing problem

Abstract

This study investigates a practical heterogeneous vehicle routing problem that involves routing a predefined fleet with different vehicle capacities to serve a series of customers to minimize the maximum routing time of vehicles. The comprehensive utilization of different types of vehicles brings great challenges for problem modeling and solving. In this study, a mixed-integer linear programming (MILP) model is formulated to obtain optimal solutions for small-scale problems. To further improve the quality of solutions for large-scale problems, this study develops a reinforcement learning-based hyper-heuristic, which introduces several meta-heuristics with different characteristics as low-level heuristics and policy-based reinforcement learning as a high-level selection strategy. Moreover, deep learning is used to extract hidden patterns within the collected data to combine the advantages of low-level heuristics better. Numerical experiments have been conducted and results indicate that the proposed algorithm exceeds the MILP solution on large-scale problems and outperforms the existing meta-heuristic algorithms.