Rolling Horizon Research Articles

Pedestrian shuttle service optimization for autonomous intersection management

Autonomous intersection management (AIM) has been widely investigated over the past two decades as a solution for signal-free intersections in a fully connected and autonomous environment. However, existing AIM approaches lack consideration for crossing pedestrians, especially in a way that enables pedestrians to cross streets safely while optimizing upcoming vehicles at the intersection to maintain traffic continuity. To address this issue, this paper proposes an Automated Pedestrian Shuttle (APS) service system to transport pedestrians across the street. Using a coupled space–time network modeling method, an optimization model, which incorporates pedestrian ride strategies and aims to minimize the total crossing time, is first established for the APS operation scheme. A Shuttle-by-Shuttle-Planning (SBSP) algorithm is developed for model solving to determine the APS operation route and the departure time from each station along the route. Then, a timing schedule optimization model is proposed to adjust the departure time of APS at each station and the time of main-lane vehicles entering an intersection to avoid vehicle conflicts. To ensure that the main-lane vehicles can enter the intersection on time, a trajectory optimization model is proposed to provide a specific driving strategy for each main-lane vehicle. A rolling horizon strategy is adopted to bridge the different optimization time windows. Finally, the effectiveness of the proposed APS service system is demonstrated through numerical simulation experiments. The results show that (1) the APS service can effectively help pedestrians cross the street with minor delays to main-lane vehicles; (2) The dynamic-route scheme is adaptable to the needs of randomly arriving pedestrians due to its demand-responsive nature; (3) The service efficiency of the system is related to the number of APSs, and the more APSs, the higher the service efficiency under the premise of unsaturation; (4) The designed SBSP algorithm significantly reduces the model-solving time compared to the commercial solver, making the APS service system potentially applicable for field implementation.

Railway crew planning with fairness over time

Passenger railway operators typically employ large numbers of drivers and guards, and are interested in providing them with fair and attractive working conditions. At Netherlands Railways (NS), the largest passenger railway operator in The Netherlands, this challenge is addressed through the use of Sharing-Sweet-and-Sour rules, which specify a fair allocation of sweet (attractive) and sour (unattractive) work over the different crew bases. While these rules are currently implemented at the crew base level and in the tactical planning phase, NS is considering formulating these rules at the individual level, in the operational planning phase, and with respect to a given planning period. This gives rise to a new problem, which we call the railway crew planning problem with fairness over time. We propose a rolling horizon approach with a penalty-based feedback mechanism and a column generation heuristic to solve this problem. On several real-life instances from NS, including up to 572 unique guards, this method is able to satisfy the individual rules for on average 95.2% of the employees.

Design and operation of modular biorefinery supply chain under uncertainty using generalized Benders decomposition

AbstractBiomass supply chain performance is heavily affected by uncertainties stemming from supply, demand, or unexpected disruptions. Unlike petrochemical plants that use crude oil, biorefineries often have to deal with the uneven spatial‐temporal distribution of feedstock supply. The modular production strategy provides more flexibility in chemical manufacturing by allowing fast capacity expansion and unit movement. However, modeling and optimizing modular biomass supply chain under uncertainties becomes challenging due to high dimensionality and the existence of discrete decisions. This work optimizes the multiperiod biomass supply chain using the rolling horizon planning and two‐stage stochastic programming framework. We then applied generalized Benders decomposition to reduce the computational complexity of the stochastic mixed integer nonlinear programming supply chain optimization. Furthermore, the solution of the stochastic programming could be used to quantitatively describe the life‐cycle assessment uncertainties of the biomass supply chain performance, demonstrating seasonality and random variability.

A hybrid deep learning approach to integrate predictive maintenance and production planning for multi-state systems

This paper develops a data-driven approach to dynamically integrate tactical production and predictive maintenance planning for a multi-state system composed of several series-parallel machines. The objective is to determine an integrated lot-sizing and preventive maintenance strategy that will minimize the sum of maintenance and production costs, while satisfying the demand for all products over the entire horizon. A rolling horizon planning strategy is adopted to continuously update the production and maintenance plans based on new data obtained through sensors. Unlike the existing integrated models, we develop a hybrid deep learning (DL) approach to coordinate maintenance and production decisions for a multi-state system composed of multiple machines. To accurately predict the health condition of each machine, the developed hybrid DL method combines the powers of convolutional neural network (CNN), long-short-term memory (LSTM), and attention technique. We use multi-state reliability theory to estimate the production capacity. Furthermore, a genetic algorithm is developed to solve large-scale problems. Benchmarking data are used to compare the results of our data-driven approach with a model-based approach, a pure LSTM, and a CNN-LSTM approach. This comparison is based on prediction accuracy, solution quality, and computational time. The obtained results show the superiority of the suggested CNN-LSTM-attention data-driven framework integrating maintenance and production.

Forecasting and planning for a critical infrastructure sector during a pandemic: Empirical evidence from a food supply chain

The meat supply chain (MSC) – a key constituent of the ‘Food & Agriculture’ CISA critical infrastructure sector, was among the most impacted by the COVID-19 pandemic. The witnessed successive demand and supply shocks uncovered the fragility of the MSC and revealed that more attention should be given by researchers and practitioners to ensure effective planning of such a critical infrastructure sector during periods of turbulence. To that end, in this paper we propose a two-stage approach for the planning of an MSC. In the first stage, we identify the most suitable model for predicting the demand and the supply. In the second stage, a multi-period multi-product mixed integer programming (MIP) model accounting for key MSC features is devised to deal with the planning of the MSC. Furthermore, in order to validate our theoretical proposition, a case study pertaining to a real-life MSC was used during the second and first wave of COVID-19 under different conditions. In particular, the results show that accurate demand and supply forecasting, and the recourse to rolling horizon planning approach, allow for satisfying the demand and maintaining the MSC profit in periods of turbulence, and so can be considered as levers for supply chain resilience.

Demand-Responsive Transit Service With Soft Time Windows Considering Real-Time Disruptions Based on Bounded Rationality

Demand-responsive transit (DRT) with smartphone-based applications is emerging as a flexible and sustainable mobility service, transforming urban transportation. Nevertheless, to satisfy the real-time and inconsistent demand, it is becoming increasingly important to capture the decision-making psychology of order cancellations. In this study, a two-phase optimization framework is presented in response to real-time disruptions, including order cancellations and the insertion of new real-time passengers. In contrast to random real-time demand, this paper is more concerned about the impacts of the feedback information on order cancellations. Bounded rationality is incorporated into the model to discuss the decision-making process of cancellation behaviors. With regard to the soft window, a compensation strategy is proposed to promote the profit while encouraging passengers for a long-term use. Additionally, solution algorithm based on variable neighborhood search (VNS) and rolling horizon is constructed to approach the Pareto solutions set. To testify the validity of the proposed algorithm, small-scale experiments in simplified Sioux Falls network are investigated for multiple runs. Meanwhile, a real-world case study in Beijing is explored to evaluate the system performance considering real-time disruptions. The results indicate that the dynamic DRT service can substantially improve the system profit but increase the penalty cost. The profit presents a significant improvement to 940 (renminbi) RMB as a result of the insert of real-time passengers. This study, therefore, not only provides a deeper insight into the analysis of passenger cancellation behavior but also contributes to construct a more flexible DRT service.

Inner-outer layer co-optimization of sizing and energy management for renewable energy microgrid with storage

Conventional microgrid optimization schemes fall short in achieving global optimality for both sizing and scheduling aspects. In response to the demand for simultaneous optimization, this paper presents a novel inner-outer layer framework that includes an outer layer dedicated to sizing optimization and an inner layer focused on Energy Management System (EMS) operational optimization. This approach has a distinctive mechanism of two-layer interaction, where the outer layer focuses on the size assignment and renewal, while the inner layer is constrained by the size assignment and provides feedback on the corresponding operational cost to the outer layer. It enhances the capability of global optimization since the optimal sizes are searched within a fixed space. To establish the range of initial generation values, the framework commences with a pre-optimization phase. Based on the convexity nature of optimization problems, a combined solution methodology involving Rolling Horizon Optimization (RHO) and Particle Swarm Optimization (PSO) is implemented. The case study demonstrates that the algorithm exhibits a good convergence within 30 generations, reaching a global satisfactory solution with a population size of 50. Furthermore, our analysis of optimal sizes across diverse scenarios fills the research gap in understanding the relationship between optimal sizes and operational scenarios. These results underscore that the impact of the time-of-day tariff strategy is significant, and it is risky to determine the optimal sizes by rule-based strategies. Finally, the comparison to other co-optimization methods shows that the proposed framework can reduce the annual total cost by up to 23.1%, which highlights its greater capability of global optimization.

A multi-level production-inventory-distribution system under mixed make to stock, make to order, and vendor managed inventory strategies: An application in the pulp and paper industry

In today's competitive environment, most companies tend to adopt a hybrid manufacturing strategy to provide more reliable services to a wide range of customers. Based on a real-world case study, we investigate a customer-wise production-inventory-distribution system in a multi-level supply chain. Within this system, at a tactical planning level, the decision concerning the production strategies including make to stock (MTS), make to order (MTO), or vendor managed inventory (VMI) strategy, to choose for each customer is made, and then at the operational planning level, the production, inventory, and distribution activities are coordinated according to the strategy assigned to the customers. The objective is to optimize the total cost while the VMI customers are timely satisfied and the target service level for MTO customers is respected. Furthermore, the purchase acceptance rate for MTS customers can be controlled to enhance capacity utilization. We develop a rolling horizon planning approach to deal with the dynamics of customers' demands under different forecasting scenarios. We perform numerical experiments under both static and rolling horizons based on adapted data from a real-world application from a Canadian pulp and paper industry. The results reveal that customers placing smaller orders are classified within the VMI strategy to mitigate potential delays. Conversely, MTS customers are predominantly characterized by larger order quantities. A sensitivity analysis of key problem parameters demonstrates that the backorder cost for MTO customers can effectively influence the strategy allocation. Moreover, our results underscore the notable influence of sales prices on the distribution of customers among strategies.

Dynamic joint decision of matching parameters and relocation strategies in ride-sourcing systems interacting with traffic congestion

As the ride-sourcing market expands, ride-sourcing fleets have increased urban traffic congestion, and in turn, road congestion is affecting ride-sourcing operations. It is crucial to incorporate this interaction mechanism into the operational decision of ride-sourcing systems. This mechanism can be described in detail by simulation models. Applying simulation-based optimization methods to the real-time operation of ride-sourcing systems is challenging due to the limitations of simulation efficiency. To tackle this issue, this study first proposes a modular simulation model combining time-driven and event-driven mechanisms. The simulation model comprises two layers: the traffic flow module at the bottom, defined by trip-based and multi-region macroscopic fundamental diagrams and cell transmission models, and the ride-sourcing operation module at the top, which considers the interaction between the traffic congestion and ride-sourcing operation and models the random behavior of passengers and drivers. We verify the efficiency and accuracy of the simulation model. The simulation model is utilized as an evaluator of the objective functions and performance measures. Then, we integrate the Bayesian optimization, parallel sampling, and rolling horizon approaches to develop an efficient and effective simulation-based optimization framework for the dynamic joint decision-making of matching radius, matching intervals, and threshold relocation strategies. Six groups of experiments for the traffic scenario in Berlin reveal some interesting findings. (1) The matching rate first increases and then decreases with the matching radius (a concave function), while the closest study does not think so. (2) As the fleet size grows, passenger waiting time reaches its minimum and then rises because of the increased endogenous congestion generated by ride-sourcing fleets. (3) The optimal operation strategy considering endogenous congestion not only leads to an increase in the system revenue and efficiency but also leads to congestion relief, even though the latter was not our initial objective. This benefit is the most pronounced when the fleet size and background traffic are moderate.

Multilayered Energy Management Framework for Extreme Fast Charging Stations Considering Demand Charges, Battery Degradation, and Forecast Uncertainties

To achieve a cost-effective and expeditious charging experience for extreme fast charging station (XFCS) owners and electric vehicle (EV) users, the optimal operation of XFCS is crucial. It is however challenging to simultaneously manage the profit from energy arbitrage, the cost of demand charges, and the degradation of a battery energy storage system (BESS) under uncertainties. This paper, therefore, proposes a multi-layered multi-time scale energy flow management framework for an XFCS by considering long- and short-term forecast uncertainties, monthly demand charges reduction, and BESS life degradation. In the proposed approach, an upper scheduling layer (USL) ensures the overall operation economy and yields optimal scheduling of the energy resources on a rolling horizon basis, thereby considering the long-term forecast errors. A lower dispatch layer (LDL) takes the short-term forecast errors into account during the real-time operation of the XFCS. Per the latest research, monthly demand charges can be as high as 90% of the total monthly bills for EV fast charging stations; to this end, this paper takes the first attempt at the reduction of demand charges cost by considering the trade-off between the energy cost and monthly demand charges. Contrasting literature, this work allocates an energy reserve in the BESS stored energy to deal with the impact of short-term forecast errors on the optimized real-time operation of the XFCS. Moreover, degradation modeling considers the trade-off between short-term benefits and long-term BESS life degradation. Lastly, case studies and a comparative analysis prove the efficacy of the proposed framework.

Cluster stability-driven optimization for enhanced routing in heterogeneous vehicular networks

The new era of the Internet of Things is promoting the evolution of self-driving vehicles into connected and autonomous vehicles (CAVs). The deployment of CAVs in smart cities is highly dependent on the performance of their underlying networks known as vehicular networks. Designing an effective clustering approach is of great importance in such dynamic networks as it can significantly improve the reliability and scalability of the routing protocols. In this paper, we consider a heterogeneous vehicular network architecture that supports vehicle-to-vehicle and vehicle-to-infrastructure communications based on IEEE 802.11p and cellular networks (LTE/5G) with direct communications, namely cellular vehicle-to-everything (C-V2X) technologies. We introduce a novel clustering scheme for real-time routing based on the proposed network architecture. We formulate the problem of optimal clustering for connected and autonomous vehicles (OCCAV) and show that the problem is NP-hard. We then develop a clusters' stability maximization algorithm (CSM), which utilizes the stability degree of vehicles over a prediction horizon to efficiently solve the optimization problem in real-time. The algorithm is used within a rolling horizon framework for continuously solving the problem, making the resulting clusters adaptive to future traffic dynamics. We propose a hybrid routing protocol based on our clustering scheme, aiming to improve the packet delivery ratio and reduce the average delivery delay. For evaluation purposes, we use extensive realistic simulations based on mobility scenarios validated using real vehicular trajectories. The results demonstrate that our clustering scheme improves the alternative algorithms in terms of the average cluster head duration, cluster head change rate, cluster member duration, and overall stability by 61%, 62%, 44%, and 52%, respectively. CSM also outperforms clustering overhead by 54%. Compared to the other cluster-based routing, our scheme also achieves a higher packet delivery ratio by up to 30%, and a lower average delay by up to 52%. To provide an in-depth analysis of the optimality of our scheme and its alternatives, we also use the Gurobi optimizer to find an optimal solution to the OCCAV problem. The results suggest that our scheme can achieve near-optimal cluster stability.

Navigating Efficiency and Uncertainty: Risks of Relying on an At-Will Workforce in Urban Meal Delivery

Increasing popularity in gig employment has enabled the use of an at-will workforce of self-contracted couriers to participate in many service industries serving urban areas. This gig workforce has come to play a particularly important role in the growing meal delivery service industry. Hiring at-will couriers for delivery job fulfillment can decrease the costs of satisfying nonstationary demand. However, at-will workers can show up for work at their will and without notice. Thus, this puts the service performance of the delivery company that relies on effective workforce management to ensure timely delivery of orders at risk. This work investigates the tradeoffs between using such an at-will workforce of couriers in place of a fixed fleet of drivers in servicing a meal delivery environment. A stochastic DES with tabu search heuristic and embedded ejection chain approach for optimal delivery job bundling, routing, and assignment was developed and run within a rolling horizon framework to replicate the dynamics of the meal delivery setting. Condition Value at Risk (CVaR) is adopted to measure the risk of late delivery due to uncertainty in workforce availability. Results from a numerical case study with 25 restaurants and 613 orders arriving over a 14-h period show tradeoffs from using at-will couriers in place of a comparable fixed fleet of drivers in terms of delivery resource utilization, efficiency risk of failing to satisfying orders and risk of significantly late delivery. Results indicate that using at-will couriers for meal delivery can enable more efficient use of delivery resources, but at the cost of a higher risk of late delivery, and sometimes intolerably late delivery, as compared to using a fixed fleet of drivers to fulfill orders.

Dynamic repositioning in bike-sharing systems with uncertain demand: An improved rolling horizon framework

The bike-sharing system (BSS) serves as a vital shared mobility mechanism, providing first- and last-mile services in urban transportation. However, one ongoing challenge in the BSS is the spatiotemporal imbalance caused by fluctuating user demand. To address this issue, this paper studies a dynamic bike repositioning problem under scenario-based demand to achieve a balance in inventory levels for electronic fences. Specifically, a multi-period two-stage stochastic model is proposed to make the trade-off between operational costs and service quality. The model incorporates a vehicle no-return (VNR) strategy, which allows vehicles to remain at the last station at the end of each period, awaiting the next repositioning task. To solve the model, an improved rolling horizon framework is introduced to effectively handle the temporal dimension complexity of the multi-period problem. Then a customized hybrid heuristic algorithm based on parallel genetic algorithm and variable neighborhood search is developed to obtain high-quality solutions. To validate the effectiveness and applicability of the proposed method, small-scale and practical-scale numerical experiments are specifically conducted based on real-world BSS data from Beijing, China. The results demonstrate the proposed method (i) creates a substantial reduction in the number of unmet demands when multiple scenarios potentially occur compared to the determined model, (ii) yields an average reduction of 29.04% in operational costs compared to the scheme without the VNR strategy, (iii) overcomes the short-sightedness inherent in the conventional single-period rolling horizon approach, exhibiting closer to optimal performance, and (iv) is superior to baseline approaches in terms of solution quality and efficiency.

Storage management in a rolling horizon Dynamic Real-Time Optimization (DRTO) methodology for a non-concentrating solar thermal plant for low temperature heat production

The intermittency and uncertain forecasts of solar irradiation complicate the operation of a solar thermal plant with thermal storage for heat production. In this work, a rolling horizon Dynamic Real-Time Optimization (DRTO) methodology is proposed to determine the economic optimal operation of a non-concentrating solar thermal plant for low temperature heat production, using a planning phase to improve storage management. The methodology is tested online on a detailed simulation model representing a large-scale solar thermal plant in case studies, using variable heat demand and real data for the weather forecasts and measurements. It was shown that DRTO performs better than offline Dynamic Optimization thanks to the use of updated weather forecasts and the regular re-initialization of the system state using measurements. An increase up to 57% in supplied energy and a decrease up to 35% in operating costs were achieved with DRTO compared to DO. Guidelines on the best storage management policy at the DRTO level, using the planning phase, are formulated. While storing the maximum energy possible for later use is the best option when overheating is not a risk, following the planned storage state helps to prevent overheating when the solar irradiation is high and the heat demand low.

Maximizing the survival probability in a cash flow inventory problem with a joint service level constraint

This paper investigates a multi-period stochastic cash flow inventory problem with the aim of maximizing the long-term survival probability, which may be the objective of some retailers especially in periods of economic distress. Demand in each period is stochastic and can be non-stationary. In order to avoid too many lost sales under this objective, we introduce a joint chance constraint requiring the probability of no stockouts during the planning horizon to be higher than a specified service level. We develop a scenario-based model and a sample average approximation (SAA) model to solve the problem. A statistical upper bound on the survival probability based on SAA is provided and we discuss upper and lower bounds for the problem based on stochastic dynamic programming. We also propose a rolling horizon approach with service rate updating to test the out-of-sample performance of the two stochastic models and solve problems with long planning horizons. We test the two methods in large numerical tests and find that the rolling horizon approach together with the stochastic models can solve realistically sized problems in reasonable time.

A Mixed‐Integer Linear Programming Framework for Optimization of Water Network Operations Problems

AbstractWater distribution systems (WDSs) are critical infrastructure used to convey water from sources to consumers. The mathematical framework governing the distribution of flows and heads in extended period simulations of WDSs lends itself to application in a wide range of optimization problems. Applying the classical mixed integer linear programming (MILP) approach to model WDSs hydraulics within an optimization framework can contribute to higher solution accuracy with lower computational effort. However, adapting WDSs models to conform to a MILP formulation has proven challenging because of the intrinsic non‐linearity of system hydraulics and the complexity associated with modeling hydraulic devices that influence the state of the WDS. This paper introduces MILPNet, an adjustable framework for WDSs that can be used to build and solve an extensive array of MILP optimization problems. MILPNet includes constraints that represent the mass balance and energy conservation equations, hydraulic devices, control rules, and status checks. To conform to MILP structure, MILPNet employs piece‐wise linear approximation and integer programming. MILPNet was implemented and tested using Gurobi Python API. Modeling accuracy was shown to be comparable to EPANET, a public domain software for hydraulic modeling, and sensitivity analyses were conducted to examine the impacts of the modeling assumptions on the performance of MILPNet. Additionally, application of the framework was demonstrated using pump scheduling optimization examples in single and rolling horizon scenarios. Our results show that MILPNet can facilitate the construction and solution of optimization problems for a range of applications in WDSs operations.

Collision-Free Merging Control Via Trajectory Optimization for Connected and Autonomous Vehicles

On-ramp merging zones have long been accident-prone spots as well as major triggers of traffic congestion in freeway networks. This paper proposes a collision-free merging control for connected and autonomous vehicles based on trajectory optimization to facilitate traffic safety and efficiency. The optimization is embedded in a model predictive control algorithm with the aim of maximizing the average travel speeds of all vehicles in the planning horizon. To guarantee collision-free merging, the merging control allows only one vehicle to pass the merging zone at a time. Safety constraints, including minimum time gaps and minimum distances between vehicles, are also considered. A rolling horizon control framework is established to enable dynamic traffic control; specifically, generating optimal acceleration decisions for each time step. The optimization problem is formulated as mixed-integer linear programming to enable efficient computation. Numerical studies and sensitivity analysis were conducted to validate the effectiveness and applicability of the proposed method. The results show that the introduced safety constraints can guarantee collision-free merging. Compared to the traditional method, Ih is built on the intelligent driver model and the minimizing overall braking induced by lane-change model, the proposed optimization-based approach results in 5.0%–22.2% improvements with respect to average travel speeds during the merging process. The safety and efficiency improvements are more significant when dealing with higher demand levels. Notably, compared to the difference in initial speeds between the mainline and on-ramp vehicles, the length of the acceleration lane has a greater impact on the control performance.

Continuity of care in home health care scheduling: a rolling horizon approach

Continuity of care in home health care scheduling: a rolling horizon approach

Integration of cascaded coordinated rolling horizon control for output power smoothing in islanded wind–solar microgrid with multiple hydrogen storage tanks

This paper presents a strategy based on the hierarchical rolling horizon control, also called model predictive control (MPC), for efficiently managing a hydrogen-energy storage system (HESS) within an islanded wind–solar microgrid. An electrolyzer uses electricity generated from renewable sources to produce clean hydrogen, which is then re-electrified by a fuel cell as needed to meet the microgrid’s loads. The main contribution lies in the incorporation of multiple hydrogen storage tanks in the HESS, distinguishing it from existing literature, which typically focuses on a single tank. The incorporation of multiple tanks in the HESS enables the storage of large volumes of hydrogen for long-term use, allowing the microgrid to operate autonomously without interaction with the utility grid. In order to ensure optimal performance, the selection of the most suitable device for operation at each time-step is crucial. The proposed control strategy takes into account the economic and operational costs, degradation aspects, and physical constraints of the HESS, while simultaneously ensuring the tracking of reference demands and with the highest priority smoothing out the variations of renewable energy sources. Numerical simulations and a lab-scale microgrid setup demonstrate that the controller effectively manages the HESS thus satisfying economic constraints and optimizing device costs, even when deviations occur between the predicted and real-time scenarios. Furthermore, the inclusion of multiple hydrogen tanks allows the microgrid to both mitigate fluctuations in renewable power sources and effectively meet load demand.

The stochastic share-a-ride problem with electric vehicles and customer priorities

We introduce a stochastic share-a-ride problem in which a fleet of electric vehicles (EV) in a ride-hailing system are dynamically dispatched to serve passenger and parcel orders in a shared manner. We assume uncertain demands of both passenger and parcel orders and consider that passenger orders have priority over parcel orders. Passengers must be transported directly from their origins to destinations, while parcels can share a vehicle with other orders. The operator of the ride-hailing platform needs to decide whether to accept a newly arrived service request, how to assign orders to vehicles, and how to route and charge the EVs. To develop dynamic policies for the problem, we formulate the problem as a Markov decision process (MDP) and propose a reinforcement learning (RL) approach to solve the problem. We develop action-space restriction and state-space aggregation schemes to facilitate the implementation of the RL algorithm. We also present two rolling horizon heuristic methods to develop dynamic policies for our problem. We conduct computational experiments based on real-world taxi data from New York City. The computational results show that our RL policies perform better than the three benchmark policies in terms of serving more orders and collecting more rewards. Our RL policies are able to make high-quality decisions more efficiently when compared with the rolling horizon policies.