Graph attention reinforcement learning with flexible matching policies for multi-depot vehicle routing problems

Abstract

Multi-depot vehicle routing problem with soft time windows (MD-VRPSTW) is a valuable practical issue in urban logistics. However, heuristic methods may fail to generate high-quality solutions for massive problems instantly. Thus, this paper presents a novel reinforcement learning algorithm integrated with graph attention network (GAT-RL) to efficiently solve the problem. This method utilizes the encoder–decoder architecture to produce routes for vehicles starting from different depots iteratively. The encoder architecture employs graph attention network to mine the complex spatial–temporal correlations within time windows. Then, the decoder architecture designs fixed-order and full-pair matching policies to generate solutions. After off-line training, experiments show that this approach consistently outperforms Google OR-Tools with negligible computational time. Particularly, the robustness of the pre-trained model is validated under multiple sources of variations and uncertainties, including customer/depot numbers, vehicle capacities, and en-route traffic conditions.