Long-Term Matching Optimization With Federated Neural Temporal Difference Learning in Mobility-on-Demand Systems

Abstract

Efficient bipartite graph matching of orders and drivers is a central operational problem in industrial mobility-on-demand (MOD) systems. Traditional studies adopt pure combinatorial optimization models for order-and-driver matching, which do not consider the long-term rewards of the dynamic MOD decision making process. Toward long-term optimization, this article presents a systemic paradigm of online matching with federated neural temporal difference learning (FedTDLearning), which encompasses learning and matching phases. During the learning phase, the long-term matching process is modeled as a Markov decision process, which is typically solved by employing data-driven reinforcement learning in an offline central training scheme. Massive amounts of data would be generated on the network by industrial MOD systems. It is impossible to send all the large-scale industrial data to the cloud server for centralized model training due to network bandwidth limitations and safety concerns. Therefore, a generic and innovative FedTDLearning is proposed to achieve long-term matching in a distributed manner. During the matching phase, a real-time bipartite matching optimization problem is formulated to maximize the learned spatiotemporal value and minimize the pickup distance, which is reducible to the minimum-cost maximum weight bipartite graph matching problem. A distance-learned-value ratio algorithm is proposed to find an optimal matching in the bipartite graph based on the joint optimization of FedTDLearning and the combinatorial fractional programming approach. Furthermore, to pursue optimal computation efficiency, we solve the real-time matching problem by constructing a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -nearest neighbor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(k$ </tex-math></inline-formula> NN) bipartite graph where each order is connected with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> NN drivers. Using openly available real-world data, the prototype system and experimental evaluations show that our proposed algorithms have effective problem-solving capability in practice.