Order Dispatching Research Articles

A Data-Driven Spatiotemporal Simulator for Reinforcement Learning Methods

Spatiotemporal applications such as taxi order dispatching and warehouse task scheduling depend critically on the algorithms for operational efficiency. However, the inherent dynamic nature of these applications presents challenges in algorithm design. The growth of mobility services has facilitated the collection of extensive spatiotemporal data, which in turn prompted algorithm designers to use data-driven methods. Reinforcement learning (RL), recognized for its strong performance and suitability for spatiotemporal contexts, has garnered considerable research interest. Despite their potential, RL algorithms necessitate the use of a simulator for both training and validation purposes. However, no specific simulation system has been developed for spatiotemporal algorithm design. This vacancy hinders the progress of spatiotemporal algorithm designers. In this demo, we build a system called Data-driven Spatiotemporal Simulator (DSS), hoping to bring convenience for spatiotemporal algorithm designers. DSS is adept at handling problems related to taxi order dispatching and warehouse task scheduling and possesses the versatility to be expanded for other user-defined scenarios. The system includes visualization modules that offer insightful panels, alongside developer tools designed to streamline the development process. This enables designers to efficiently craft, evaluate, and refine their algorithms, potentially accelerating innovation in spatiotemporal application development.

Optimizing Long-Term Efficiency and Fairness in Ride-Hailing Under Budget Constraint via Joint Order Dispatching and Driver Repositioning

Optimizing Long-Term Efficiency and Fairness in Ride-Hailing Under Budget Constraint via Joint Order Dispatching and Driver Repositioning

NondBREM: Nondeterministic Offline Reinforcement Learning for Large-Scale Order Dispatching

One of the most important tasks in ride-hailing is order dispatching, i.e., assigning unserved orders to available drivers. Recent order dispatching has achieved a significant improvement due to the advance of reinforcement learning, which has been approved to be able to effectively address sequential decision-making problems like order dispatching. However, most existing reinforcement learning methods require agents to learn the optimal policy by interacting with environments online, which is challenging or impractical for real-world deployment due to high costs or safety concerns. For example, due to the spatiotemporally unbalanced supply and demand, online reinforcement learning-based order dispatching may significantly impact the revenue of the ride-hailing platform and passenger experience during the policy learning period. Hence, in this work, we develop an offline deep reinforcement learning framework called NondBREM for large-scale order dispatching, which learns policy from only the accumulated logged data to avoid costly and unsafe interactions with the environment. In NondBREM, a Nondeterministic Batch-Constrained Q-learning (NondBCQ) module is developed to reduce the algorithm extrapolation error and a Random Ensemble Mixture (REM) module that integrates multiple value networks with multi-head networks is utilized to improve the model generalization and robustness. Extensive experiments on large-scale real-world ride-hailing datasets show the superiority of our design.

Understanding the relationship between normative records of appeals and government hotline order dispatching: a data analysis method

PurposeAdvanced big data analysis and machine learning methods are concurrently used to unleash the value of the data generated by government hotline and help devise intelligent applications including automated process management, standard construction and more accurate dispatched orders to build high-quality government service platforms as more widely data-driven methods are in the process.Design/methodology/approachIn this study, based on the influence of the record specifications of texts related to work orders generated by the government hotline, machine learning tools are implemented and compared to optimize classify dispatching tasks by performing exploratory studies on the hotline work order text, including linguistics analysis of text feature processing, new word discovery, text clustering and text classification.FindingsThe complexity of the content of the work order is reduced by applying more standardized writing specifications based on combining text grammar numerical features. So, order dispatch success prediction accuracy rate reaches 89.6 per cent after running the LSTM model.Originality/valueThe proposed method can help improve the current dispatching processes run by the government hotline, better guide staff to standardize the writing format of work orders, improve the accuracy of order dispatching and provide innovative support to the current mechanism.

Order dispatching and vacant vehicles rebalancing for the first-mile ride-sharing problem

Given a set of transport requests to a transit station and a set of homogeneous vehicle, both geographically dispersed in a business area, the First-Mile Ride-Sharing Problem (FMRSP) consists of finding least cost vehicle routes to transport passengers to the station by shared rides. In this paper we formulate the problem as a mathematical optimization problem and study the effectiveness of preventive movements of idle vehicles (i.e., rebalancing) in order to anticipate future demand. That is, we identify promising rebalanacing locations based on historical data and give the model incentives to assign vehicles to such location. We then assess the effectiveness of such movements by simulating online usage of the mathematical model in a rolling-horizon framework. The results show that rebalancing is consistently preferable both in terms of profits and service rate. Particularly, in operating contexts where the station is not centrally located, rebalancing movements increase both profits and service rates by around 30% on average.

An adaptive agent-based approach for instant delivery order dispatching: Incorporating task buffering and dynamic batching strategies

The volume of instant delivery has witnessed a significant growth in recent years. Given the involvement of numerous heterogeneous stakeholders, instant delivery operations are inherently characterized by dynamics and uncertainties. This study introduces two order dispatching strategies, namely task buffering and dynamic batching, as potential solutions to address these challenges. The task buffering strategy aims to optimize the assignment timing of orders to couriers, thereby mitigating demand uncertainties. On the other hand, the dynamic batching strategy focuses on alleviating delivery pressure by assigning orders to couriers based on their residual capacity and extra delivery distances. To model the instant delivery problem and evaluate the performances of order dispatching strategies, Adaptive Agent-Based Order Dispatching (ABOD) approach is developed, which combines agent-based modelling, deep reinforcement learning, and the Kuhn-Munkres algorithm. The ABOD effectively captures the system's uncertainties and heterogeneity, facilitating stakeholders learning in novel scenarios and enabling adaptive task buffering and dynamic batching decision-makings. The efficacy of the ABOD approach is verified through both synthetic and real-world case studies. Experimental results demonstrate that implementing the ABOD approach can lead to a significant increase in customer satisfaction, up to 275.42%, while simultaneously reducing the delivery distance by 11.38% compared to baseline policies. Additionally, the ABOD approach exhibits the ability to adaptively adjust buffering times to maintain high levels of customer satisfaction across various demand scenarios. As a result, this approach offers valuable support to logistics providers in making informed decisions regarding order dispatching in instant delivery operations.

Online acceptance probability approximation in peer-to-peer transportation

Crowdsourced transportation by independent suppliers (or drivers) is central to urban delivery and mobility platforms. While utilizing crowdsourced resources has several advantages, it comes with the challenge that suppliers are not bound to assignments made by the platforms. In practice, suppliers often decline offered service requests, e.g., due to the required travel detour, the expected tip, or the area a request is located. This leads to inconveniences for the platform (ineffective assignments), the corresponding customer (delayed service), and also the suppliers themselves (non-fitting assignment, less revenue). Therefore, the objective of this work is to analyze the impact of a platform approximating and incorporating individual suppliers’ acceptance behavior into the order dispatching process and to quantify its impact on all stakeholders (platform, customers, suppliers). To this end, we propose a dynamic matching problem where suppliers’ acceptances or rejections of offers are uncertain. Suppliers who accept an offered request are assigned and reenter the system after service looking for another offer. Suppliers declining an offer stay idle to wait for another offer, but leave after a limited time if no acceptable offer is made. Every supplier decision reveals only their acceptance or rejection information to the platform, and in this paper, we present a corresponding mathematical model and an approximation method that translates supplier responses into updated approximations of the likelihood of a specific supplier to accept a specific future offer and use this information to optimize subsequent offering decisions. We show via a computational study based on crowdsourced food delivery that online approximation and incorporating individual supplier acceptance estimates into order dispatching leads to overall more successful assignments, more revenue for the platform and most of the suppliers, and less waiting for the customers to be served. We also show that considering individual supplier behavior can lead to unfair treatment of more agreeable suppliers.

Combinatorial Optimization Meets Reinforcement Learning: Effective Taxi Order Dispatching at Large-Scale

Ride hailing has become prevailing. Central in ride hailing platforms is taxi order dispatching which involves recommending a suitable driver for each order. Previous works use pure combinatorial optimization solutions for taxi dispatching, which suffer in practice due to complex dynamics of demand and supply and temporal dependency among dispatching decisions. Recent studies try to adopt data-driven method into combinatorial optimization hoping knowledge from history data would help overcome these challenges. Among these attempts, adoption of reinforcement learning shows great promise but current adoptions are a unidirectional integration which restricts the potential performance gains. In this work, we propose \underline{L}earning \underline{T}o \underline{D}ispatch(\algoname), a systematic solution that allows synergic integration of reinforcement learning and combinatorial optimization for large-scale taxi order dispatching. We demonstrate the necessity of online learning and taxi scheduling for reinforcement learning to work in synergy with combinatorial optimization, and devise corresponding algorithms. We also devise many tricks for more efficient calculation of the bipartite matching. Experiments show our methods can improve 36.4% and 42.0% on utility and efficiency at most, respectively. Especially, it achieves state-of-the-art performance in terms of utility.

Region-Aware Hierarchical Graph Contrastive Learning for Ride-Hailing Driver Profiling

Driver profiling, which is the process of extracting driver preferences and behavioral patterns from collected driving data, can be performed on a microscopic or macroscopic scale. Microscopic driver profiling, which uses onboard data, can be incorporated into advanced driver assistance systems or used by insurance companies to implement pay-as-you-drive programs. However, transportation network companies (TNCs) typically cannot access onboard data owing to privacy and cost concerns. Therefore, TNCs typically perform macroscopic driver profiling using the raw GPS trajectories generated by smartphones. Accurate profiling of ride-hailing drivers can enhance the user experience and order dispatching process by allowing the TNCs to accurately predict various downstream tasks such as time of arrival of rides. Notably, most of the approaches that use raw GPS trajectories for driver profiling directly leverage trajectory data without any comprehensive analysis, resulting in neglect of the rich regional semantic information and underlying correlations among trajectories. Therefore, we study the ride-hailing driver profiling problem and propose a Hierarchical Graph Contrastive Learning (HGCL) framework that can automatically learn low-dimensional embeddings encoding driver behaviors from raw GPS data. The HGCL framework consists of a hierarchical graph neural network for capturing the regional features of trajectories and a hierarchical contrastive learning strategy aimed at learning high-quality representations at different levels. The effectiveness of the proposed model is evaluated using driver representation embeddings learned from a real-world large-scale dataset for three downstream tasks. The results of extensive experiments demonstrate the efficacy of the proposed HGCL framework for driver profiling.

Order Dispatching in Ride-Pooling with Walking Points Search

Ride-pooling has significantly enhanced the system efficiency of current on-demand ride-sharing services. However, as the numbers of on-board passengers increase, more detours inevitably occur since ride-pooling provides door-to-door service for everyone. To solve this problem, we focus on rider-participating dispatch by searching walking points, equivalent to alternative pick-up points from origins and alternative drop-off points from destinations. Based on the existing framework for large-scale ride-pooling, we develop our walking point search algorithm, which finds cost-minimizing alternatives. In addition, our approach enables the model to reflect the sensitivity of riders to given walking points by introducing the probability of riders’ acceptance. We conduct a simulation with the Yellow Cap Taxi data set in New York City to validate and compare with the base model, which does not include walking. The results show an increase from 69.56% to 77.84% in the service rate, an improvement of 18.2% in delay time, and 8.6% in in-vehicle time. With the increased service rate, the average travel times of vehicles are reduced by 1.5%, allowing drivers to spend more time rebalancing. Furthermore, we show that the effect of walking is maximized in high-demand areas during peak hours. This study demonstrates that walking can substantially enhance operational efficiency, mitigating the supply-demand imbalance with limited fleets. The proposed model can also be used in optimizing the meeting points for various high-capacity vehicles, such as on-demand shuttles.

Model-Based Vehicle-Miles Traveled and Emission Evaluation of On-Demand Food Delivery Considering the Impact of COVID-19 Pandemic

The market for on-demand food delivery (ODFD) has increased considerably, especially during the COVID-19 pandemic. It is crucial for transportation and environmental agencies to understand how ODFD has reshaped the travel patterns of people, affecting vehicle-miles traveled (VMT) as well as pollutant emissions in the transportation system. However, the lack of public data from food delivery companies makes it challenging to quantify the impact of on-demand delivery on the real-world transportation network. In this research, we propose a comprehensive framework to quantify the VMT and emissions incurred by ODFD with three main components: (i) a daily activity generation tool, Comprehensive Econometric Micro-simulator for Daily Activity-travel Patterns, to create a simulation scenario of ODFD behaviors based on a real-world roadway network and population demographics in the City of Riverside, California; (ii) an efficient order dispatching and routing algorithm, adaptive large neighborhood search, to obtain a high quality order dispatching and routing plan; (iii) an emission evaluation model, emission factor (EMFAC), to evaluate pollutant emissions from all dining-related trips. Both short-term and long-term impacts of the COVID-19 pandemic are evaluated. Experimental results show that ODFD has great potential to reduce the dining-related VMT and emissions. The total dining-related VMT in the during-pandemic case decreased by 38% and in the after-pandemic case reduced by 6% to 9%, and the corresponding environmental impacts were reduced accordingly. Meanwhile, emissions reduced significantly with more electric vehicles involved in food delivery. With 100% electric delivery fleet, the ODFD service can save 14% to 22% of emissions after the COVID-19 pandemic.

TransFloor

Smart on-demand delivery services require accurate indoor localization to enhance the system-human synergy experience of couriers in complex multi-story malls and platform construction. Floor localization is an essential part of indoor positioning, which can provide floor/altitude data support for upper-level 3D indoor navigation services (e.g., delivery route planning) to improve delivery efficiency and optimize order dispatching strategies. We argue that due to label dependence and device dependence, the existing floor localization methods cannot be flexibly deployed on a large scale in numerous multi-story malls across the country, nor can they apply to all couriers/users on the platform. This paper proposes a novel self-evolving and user-transparent floor localization system named TransFloor, based on crowdsourcing delivery data (e.g., order status and sensors data) without additional label investment and specialized equipment constraints. TransFloor consists of an unsupervised barometer-based module--IOD-TKPD and an NLP-inspired Wi-Fi-based module--Wifi2Vec, and Self-Labeling is a perfect bridge between both to completely achieve label-free and device-independent floor positioning. In addition, TransFloor is designed as a lightweight plugin embedded into the platform without refactoring the existing architecture, and it has been deployed nationwide to adaptively launch real-time accurate 3D/floor positioning services for numerous crowdsourcing couriers. We evaluate TransFloor on real-world records from an instant delivery platform (involving 672,282 orders, 7,390 couriers, and 6,206 merchants in 388 malls during two months). It can achieve an average accuracy of 94.61% and demonstrate good robustness to device heterogeneity and adaptive durability, outperforming existing state-of-the-art methods. Crucially, it can effectively improve user satisfaction and reduce overdue delivery by providing accurate floor navigation information in complex multi-story malls. As a case study, the platform reduces erroneous order scheduling by 60% and overdue delivery by 2.7%, and increases delivery efficiency by reducing courier arrival time by 12.27 seconds accounting for 7.29%. We believe that the key ideas of TransFloor can be extended to other crowdsourcing scenarios for the public further.

Dynamic Order Dispatching With Multiobjective Reward Learning

Traffic supply-demand mismatching has a severe impact on intelligent transportation systems. Fortunately, order dispatching is a promising option to mitigate the traffic supply-demand imbalance. Along this line, this article proposes the Multi-Driver Multi-Order Dispatching (MDMOD) method to make efficient order dispatching policy and enhance the experience of drivers and passengers. In the proposed MDMOD method, the Dynamic Multi-Objective Reward Learning (DMRL) algorithm is proposed to measure the driver-order-pair value, which illustrates the importance of a driver serving a specific order. A centralized matching algorithm is introduced to match all drivers and orders to maximize all driver-order-pair values. The multi-objective reward in the DMRL algorithm considers both immediate gains (i.e., pick-up distance) and future gains (i.e., the future traffic demand of order destination) to effectively improve the experience of drivers and passengers. Furthermore, by introducing the driver service level into the multi-objective reward, the “outstanding driver better reward” mechanism is realized to promote the ecological development of ride-sharing platforms. Notably, the Temporal-Graph Convolutional Network algorithm is proposed to predict the future traffic demand. Some virtual orders, which generated with the predicted future traffic demand, are dispatched to idle drivers to multiplex the traffic supply fully. A simulator is designed to test the performance of the proposed MDMOD method, experimental results demonstrate that the MDMOD method outperforms the state-of-the-art methods in terms of Average Driver Income and Order Response Rate.

Opportunistic maintenance scheduling with deep reinforcement learning

The great complexity of advanced manufacturing processes combined with the high investment costs for manufacturing equipment makes the integration of maintenance scheduling a challenging, but similarly crucial task. Opportunistic maintenance scheduling holds the potential to increase the operational performance by considering the opportunity cost of maintenance measures. At the same time, reinforcement learning (RL) has proved to be able to handle complex scheduling tasks. Therefore, RL constitutes a promising approach to develop an integrated maintenance scheduling model to consider order dispatching and maintenance scheduling in a single decision support system. This paper models a real-world use case of a semiconductor front-end wafer fabrication by using discrete-event simulation. In the simulated scenarios, the performance of the integrated dispatching and maintenance scheduling is regulated by both complex novel heuristics adapted to opportunistic maintenance and reinforcement learning. The results show that the RL policy is able to learn a competitive joint scheduling strategy by including internal and external opportunistic opportunities. This indicates that opportunistic maintenance scheduling, with and without RL, holds the potential to improve the performance not only of semiconductor manufacturing but capital-intense machinery industry alike.

Modeling stochastic service time for complex on-demand food delivery

Uncertainty is everywhere in the food delivery process, which significantly influences decision-making for complex on-demand food delivery problems, affecting delivery efficiency and customer satisfaction. Especially, the service time is an indispensable part of the delivery process impacted by various uncertain factors. Due to the simplicity and high accuracy requirement, we model the uncertain service time as a Gaussian mixture model (GMM). In detail, we transform the distribution estimation problem into a clustering problem by determining the probability of each data belonging to each component (each cluster as well). A hybrid estimation of distribution algorithm is proposed to intelligently solve the clustering problem with the criterion to optimize quality and simplicity simultaneously. First, to optimize the simplicity, problem-specific encoding and decoding methods are designed. Second, to generate initial solutions with good clustering results, a Chinese restaurant process-based initialization mechanism is presented. Third, a weighted-learning mechanism is proposed to effectively guide the update of the probability model. Fourth, a local intensification based on maximum likelihood is used to exploit better solutions. The effect of critical parameters on the performances of the proposed algorithm is investigated by the Taguchi design of the experimental method. To demonstrate the effectiveness of the proposed algorithm, we carry out extensive offline experiments on real-world historical data. Besides, we employ the GMMs obtained by our algorithm in a real-world on-demand food delivery platform, Meituan, to assist decision-making for order dispatching. The results of rigorous online A/B tests verify the practical value of introducing the uncertainty model into the real-life application.

Coordinating ride-sourcing and public transport services with a reinforcement learning approach

Combining ride-sourcing and public transit services (with ride-sourcing service to address the first/last-mile issues) can bring many benefits, such as saving passengers’ trip fares, improving drivers’ earnings, reducing gas emissions, and alleviating traffic congestion. However, it still remains a challenging issue to coordinate ride-sourcing and public transit services through real-time order dispatching. In this paper, we model the order dispatching in a multi-modal transportation system as a large-scale sequential decision-making problem. A centralized algorithm is then proposed to dispatch idle drivers to arriving passenger orders and determine whether to advise passengers to use a combined mode of ride-sourcing and public transit services (if yes, the algorithm also needs to recommend an appropriate transportation hub). In particular, our proposed algorithm contains a reinforcement learning approach that estimates the long-term expected rewards, and an Integer Linear Programming (ILP) that matches idle drivers and waiting passengers in real-time based on both immediate revenue and the estimated long-term rewards. By evaluation on the real-world on-demand data and metro system in Manhattan, the proposed method shows remarkable improvement on the system’s efficiency under different density of supply and demands.

Fleet Management and Charging Scheduling for Shared Mobility-on-Demand System: A Systematic Review

Challenged by urbanization and increasing travel needs, existing transportation systems call for new mobility paradigms. In this article, we present the fleet management and charging scheduling of a shared mobility-on-demand system, whereby electric vehicle fleets are operated by a centralized platform to provide customers with mobility service. We provide a comprehensive review of system operation based on the operational objectives. The fleet scheduling strategies are categorized into four types: i) order dispatching, ii) order-dispatching and rebalancing, iii) order-dispatching, rebalancing and charging, and iv) extended. Specifically, we first identify mathematical modeling techniques implemented in the transportation network, then analyze and summarize the solution approaches including mathematical programming, reinforcement learning, and hybrid methods. The advantages and disadvantages of different models and solution approaches are compared. Finally, we present research outlook in various directions.

SmartLOC

On-demand delivery is a rapidly developing business worldwide, where meals and groceries are delivered door to door from merchants to customers by the couriers. Couriers' real-time localization plays a key role in on-demand delivery for all parties like the platform's order dispatching, merchants' order preparing, couriers' navigation, and customers' shopping experience. Although GPS has well solved outdoor localization, indoor localization is still challenging due to the lack of large-coverage, low-cost anchors. Given the high penetration of smartphones in merchants and frequent rendezvous between merchants and couriers, we employ merchants' smartphones as indoor anchors for a new sensing opportunity. In this paper, we design, implement and evaluate SmartLOC, a map-free localization system that employs merchants' smartphones as anchors to obtain couriers' real-time locations. Specifically, we design a rendezvous detection module based on Bluetooth Low Energy (BLE), build indoor shop graphs for each mall, and adopt graph embedding to extract indoor shops' topology. To guarantee anchors' accuracy and privacy, we build a mutual localization module to iteratively infer merchants' state (in-shop or not) and couriers' locations with transformer models. We implement SmartLOC in a large on-demand delivery platform and deploy the system in 566 malls in Shanghai, China. We evaluate SmartLOC in two multi-floor malls in Shanghai and show that it can improve the accuracy of couriers' travel time estimation by 24%, 43%, 70%, and 76% compared with a straightforward graph solution, GPS, Wi-Fi, and TransLoc.

Exploring multi-homing behavior of ride-sourcing drivers via real-world multiple platforms data

Multi-homing behavior refers to the behavior that ride-sourcing drivers simultaneously register and sequentially provide services on multiple ride-sourcing platforms. The multi-homing behavior of ride-sourcing drivers significantly impacts the competition among multiple ride-sourcing platforms in a competitive market. To better understand the multi-homing behavior, we present exploratory evidence on the factors that influence drivers' platform switching behavior. The RF-MNL (random forest multinomial logistic regression) framework is applied to analyze multi-homing driver behavior in a competitive ride-sourcing market. Multinomial logistic regression (MLR) is adopted to model the platform switching behavior of multi-homing drivers. The random forest is employed to seek the best combination of variables for the MLR model, which is calibrated by using the one-month multi-platform ride-sourcing data in Hangzhou, China. A variety of explanatory variables that influence ride-sourcing drivers' multi-homing behavior are estimated. The results show that the driver's socio-demographic characteristics, income level, bonus income (e.g., long-distance price rise), and work time related factors (e.g., the time gap of order dispatching, and wait time) play an essential role in determining the platform switching decision. This study corroborates the evidence of significant factors that impact drivers' switching from one ride-sourcing platform to another, which can support decision-making for ride-sourcing platforms to attract drivers serving the platform exclusively. We also examine how heterogeneity in drivers' multi-homing tendencies affects the platform's policy. To our best knowledge, this paper is one of the first quantitative studies that empirically reveal the commonly observed multi-homing behavior of ride-sourcing drivers by exploring real-world city-wide data collected on multiple platforms.

Mobility-Aware Charging Scheduling for Shared On-Demand Electric Vehicle Fleet Using Deep Reinforcement Learning

With the emerging concept of sharing-economy, shared electric vehicles (EVs) are playing a more and more important role in future mobility-on-demand traffic system. This article considers joint charging scheduling, order dispatching, and vehicle rebalancing for large-scale shared EV fleet operator. To maximize the welfare of fleet operator, we model the joint decision making as a partially observable Markov decision process (POMDP) and apply deep reinforcement learning (DRL) combined with binary linear programming (BLP) to develop a near-optimal solution. The neural network is used to evaluate the state value of EVs at different times, locations, and states of charge. Based on the state value, dynamic electricity prices and order information, the online scheduling is modeled as a BLP problem where the decision variables representing whether an EV will 1) take an order, 2) rebalance to a position, or 3) charge. We also propose a constrained rebalancing method to improve the exploration efficiency of training. Moreover, we provide a tabular method with proved convergence as a fallback option to demonstrate the near-optimal characteristics of the proposed approach. Simulation experiments with real-world data from Haikou City verify the effectiveness of the proposed method.