Solving pick-up and delivery problems via deep reinforcement learning based symmetric neural optimization

Abstract

Recently, there has been a growing trend to use deep reinforcement learning (DRL) to solve NP-hard combinatorial optimization problems such as routing problem, where a policy learned by a deep neural network guides the sequential construction of solutions. Despite their success, most of the previous studies omit the symmetry between the partitions of instances and between solutions, which makes them less effective in handling the more practical scenarios with symmetric features, such as pick-up and delivery problems (PDPs). PDPs are more challenging and practical than traditional routing problems because the correlation between pick-up nodes and delivery nodes in the former is multiple one-to-one and not simple as the correlation in the latter. Besides, most DRL-based methods cannot take both computational cost and the quality of solutions into account. To resolve this issue, we propose the Symmetric Neural Optimization for Pick-up and Delivery Problems (PD-SNO) to solve the pick-up and delivery traveling salesman problem (PDTSP) and the multi-commodity one-to-one pick-up and delivery traveling salesman problem (m-PDTSP), respectively. The PD-SNO leverages the symmetries between different partitions of instances to effectively capture the relationship between symmetric node sets (i.e., pick-up nodes and delivery nodes); the PD-SNO also leverages the symmetry between different trajectories of solution to guide the construction of solutions and training of policy. We evaluate the performance of our PD-SNO, and the results show that it outperforms various representative baselines, including traditional methods (both exact and heuristic algorithms) and state-of-the-art DRL-based methods (both construction and improvement types) on synthetic datasets. Then, we further assess the generalization performance of our PD-SNO on benchmark datasets. Finally, we research the independent effectiveness of our designs through ablation studies.