Dynamic Routing Problem Research Articles

Evaluating Meta-Heuristic Algorithms for Dynamic Capacitated Arc Routing Problems Based on a Novel Lower Bound Method

Evaluating Meta-Heuristic Algorithms for Dynamic Capacitated Arc Routing Problems Based on a Novel Lower Bound Method

Dynamic collaborative truck-drone delivery with en-route synchronization and random requests

Coordinated truck and drone delivery is gaining popularity in logistics as it can greatly reduce operation costs. However, existing studies on related operations management problems typically ignore the following important features: (i) the random appearance of requests, which require operators to dynamically respond to the requests; and (ii) the decisions of optimal launch and retrieval locations for trucks and drones instead of fixed to customer locations, which can significantly impact the overall time costs. To tackle these challenges, this study investigates the dynamic collaborative truck-drone routing problem with randomly arriving requests and synchronization on routes. We model the problem as a Markov Decision Process (MDP) and solve the MDP via a reinforcement learning (RL) approach. The proposed RL approach determines: (i) whether each request should be serviced upon arrival, (ii) which truck or drone should be assigned for the request, and (iii) the optimal en-route take-off and landing positions for paired trucks and drones. We further employ a framework of decentralized learning and centralized dispatching in RL to increase performance. Numerical experiments are conducted to assess the proposed solution approach on instances generated based on both the Solomon dataset and real-world operational data of a logistics operator in Singapore over several benchmark algorithms under various battery endurance levels of drones and distinct transportation scenarios including node-based dynamic collaborative truck-drone routing problem, dynamic non-collaborative truck and drone routing problem, and dynamic vehicle routing problem. The results show that our RL solution outperforms the benchmark algorithm in total profit by an average of 28.03 %, and our en-route takeoff and landing scenario outperforms the benchmark scenarios in total profit by an average of 8.43 % in multi-day instances. Additionally, compared to the traditional node-based landing scenario, employing our en-route takeoff and landing strategy can save 0.9 h/(drone*day) of waiting time on average.

C-SPPO: A deep reinforcement learning framework for large-scale dynamic logistics UAV routing problem

Unmanned Aerial Vehicle (UAV) stands as a burgeoning electric transportation carrier, holding substantial promise for the logistics sector. A reinforcement learning framework Centralized - S Proximal Policy Optimization (C-SPPO) based on centralized decision process and considering policy entropy (S) is proposed. The proposed framework aims to plan the best scheduling scheme with the objective of minimizing both the timeout of order requests and the flight impact of UAVs that may lead to conflicts. In this framework, the intents of matching act are generated through the observations of UAV agents, and the ultimate conflict-free matching results are output under the guidance of a centralized decision maker. Concurrently, a pre-activation operation is introduced to further enhance the cooperation among UAV agents. Simulation experiments based on real-world data from New York City are conducted. The results indicate that the proposed C-SPPO outperforms the baseline algorithms in the Average Delay Time (ADT), the Maximum Delay Time (MDT), the Order Delay Rate (ODR), the Average Flight Distance (AFD), and the Flight Impact Ratio (FIR). Furthermore, the framework demonstrates scalability to scenarios of different sizes without requiring additional training.

State Aggregation and Lower Bound-Based ADP Approach for Dynamic Pick-Up Routing Problem With Capacity Constraint

State Aggregation and Lower Bound-Based ADP Approach for Dynamic Pick-Up Routing Problem With Capacity Constraint

Dynamic robot routing optimization: State–space decomposition for operations research-informed reinforcement learning

There is a growing interest in implementing artificial intelligence for operations research in the industrial environment. While numerous classic operations research solvers ensure optimal solutions, they often struggle with real-time dynamic objectives and environments, such as dynamic routing problems, which require periodic algorithmic recalibration. To deal with dynamic environments, deep reinforcement learning has shown great potential with its capability as a self-learning and optimizing mechanism. However, the real-world applications of reinforcement learning are relatively limited due to lengthy training time and inefficiency in high-dimensional state spaces. In this study, we introduce two methods to enhance reinforcement learning for dynamic routing optimization. The first method involves transferring knowledge from classic operations research solvers to reinforcement learning during training, which accelerates exploration and reduces lengthy training time. The second method uses a state–space decomposer to transform the high-dimensional state space into a low-dimensional latent space, which allows the reinforcement learning agent to learn efficiently in the latent space. Lastly, we demonstrate the applicability of our approach in an industrial application of an automated welding process, where our approach identifies the shortest welding pathway of an industrial robotic arm to weld a set of dynamically changing target nodes, poses and sizes. The suggested method cuts computation time by 25% to 50% compared to classic routing algorithms.

Waiting Strategy for the Dynamic Meal Delivery Routing Problem with Time-Sensitive Customers Using a Hybrid Adaptive Genetic Algorithm and Adaptive Large Neighborhood Search Algorithm

In this paper, we study the dynamic meal delivery routing problem (MDRP) with time-sensitive customers. The multi-objective MDRP optimization model is developed to maximize customer satisfaction and minimize delay penalty cost and riding cost. To solve the dynamic MDRP, a novel waiting strategy is proposed to divide the dynamic problem into a series of static subproblems. This waiting strategy utilizes the decision threshold to determine rerouting points based on the number of dynamic meal orders. Meanwhile, time-sensitive priority is introduced to accelerate assignment and routing decisions for orders from customers with high time sensitivity. For each static subproblem, a hybrid AGA–ALNS algorithm that incorporates the adaptive genetic algorithm and adaptive large neighborhood search is developed to improve both the global and local search capabilities of the genetic algorithm. We validate the performance of the proposed waiting strategy and the AGA–ALNS algorithm through numerical instances. In addition, managerial insights are obtained from sensitivity analysis experiments.

Integrated Optimisation Model of Daily Freight Train Scheduling and Dynamic Railcar Routing Based on a Two-Layer Space-Time Network

This paper focuses on daily freight train scheduling and dynamic railcar routing problems for rail freight transportation at the operational level. Two mixed integer linear programming models that adopted different strategies were formulated based on a continuous two-layer time-space network. We simultaneously considered the benefits of railroad company and service quality when setting the objective function. By solving the models, we can distribute the dynamic railcar flows to the train paths in the basic train timetable to obtain the daily train operation plan over a short time horizon (e.g. a day), which will be helpful for dispatchers to make decisions such as the empty railcar distribution and car routing (trip planning). Finally, we compared two models on a part of the Chinese railroad network. The results show that the second model can effectively improve the efficiency of railroad freight transportation.

Technician routing and scheduling with employees’ learning through implicit cross-training strategy

With record high talent shortages and skill mismatches around the world, this paper investigates a variant of multi-period dynamic technician and routing problem that can be modeled as a Markov decision process. To deal with the double tradeoffs between the routing and service time costs, as well as the current and future costs, we propose an approximate dynamic programming (ADP)-based cost function approximation (CFA) algorithm — the implicit cross-training strategy (ICT). A two-phase routing and scheduling heuristic is developed to account for both employees’ learning and future information, and to facilitate an efficient implementation of CFA. Extensive computational results show that ICT can provide a better solution in the current decision with a global view in comparison with the myopic strategy. In depth analysis demonstrates that ICT trains the workforce with more balanced skillsets and workloads, which ensures the flexibility of the workforce and helps buffer against the future uncertainties with substantial routing cost savings. Additionally, ICT has much more advantages in large-scale problems with more diversified service requests and randomly distributed customers.

Battery Management in Electric Vehicle Routing Problems: A Review

The adoption of electric vehicles (EVs) has gained significant momentum in recent years as a sustainable alternative to traditional internal combustion engine vehicles. However, the efficient utilization of batteries in EVs, coupled with the growing demand for sustainable transportation, has posed complex challenges for battery management in the context of electric vehicle routing problems in a broad sense, which includes vehicle routing problems, team orienteering problems, and arc routing problems, all of them using EVs. This paper presents a comprehensive review of the state-of-the-art approaches, methodologies, and strategies for battery management in each of the aforementioned optimization problems. We explore the relevant factors influencing battery performance and the interplay between routing, charging, and energy management in the context of EVs. The paper also discusses the advances in optimization algorithms, vehicle-to-grid integration, and intelligent decision-making techniques aimed at enhancing the range, reliability, and sustainability of EV operations. Key findings indicate a paradigm shift towards addressing uncertainties, dynamic conditions, and synchronization challenges inherent in large-scale and dynamic routing problems within the context of EVs that require efficient battery management.

Sequential learning based re-optimization approaches for less model-based dynamic pick-up routing problem

We address a lessmodel-based dynamic routing problem arising from home parcel pick-up service, where lessmodel-based means existing customers who dynamically request services independently following Poisson process with a stochastic rate. Overall, through an extended application of re-optimization (RO) strategy, a Markov decision process formulation and approximation dynamic programming- and Bayes' theorem-based solution approaches are proposed. Specifically, first a pool of basic policies corresponding to all possible values of the rate are developed offline via approximate value iteration. Then, Bayes' theorem-based sequential learning is designed that can sequentially update the belief about the rate's probability distribution over its possible values. Third, coupled with the updated belief, basic policies are collectively implemented in two different ways, resulting in RO approaches which involve constructions of two different online policies, i.e. a belief-weighted deterministic policy and a belief-based random policy, and their re-optimizations at decision epochs. In the numerical study, through comparison with model-based (i.e. using full knowledge of the rate) and model-free heuristics, our approaches are examined and valuable insights are obtained. Important insights include that (i) the belief-weighted deterministic policy outperforms the belief-based random policy, and further, (ii) the former is better than the latter at preserving the improvement resulting from the improved model-based policy.

Dual-decoder attention model in hierarchical reinforcement framework for dynamic crowd logistics problem with batch-matching

Dynamic routing problem with crowd-sourced driver can be challenging as the dynamically arrived crowd drivers are utilized to meet the dynamically placed customer demand. To guarantee a certain service level, the company also operates a set of employed vehicles as a backup capacity to serve the tasks for which the use of a crowd driver is not feasible or not efficient. Thus, in this paper, we study a special dynamic routing problem that arises in such an environment, in which a mix of company vehicles and dynamically arrived crowd drivers serves dynamically placed orders. The company seeks route policies for crowd drivers and company vehicles so as to minimize the total costs. We present a hierarchical reinforcement learning method that combines upper-level agent for balancing the opportunities and risks brought by delayed batch-matching and lower-level agents for route planning. The effectiveness of our approach is demonstrated through extensive numerical analyses, with the results showing the benefits derived from the upper-level agent and the designed dual-decoder architecture of lower-level agent. The solution quality, computation time, and generalization ability of the proposed approach are all proven to be superior over existing methods.

Learning efficient in-store picking strategies to reduce customer encounters in omnichannel retail

Omnichannel retailers are reinventing stores to meet the growing demand of the online channel. Several retailers now use stores as supporting distribution centers to offer quicker Buy-Online-Pickup-In-Store (BOPS) and Ship-From-Store (SFS) services. They resort to in-store picking to serve online orders using existing assets. However, in-store picking operations require picker carts traveling through store aisles, competing for store space, and possibly harming the offline customer experience. To learn picking policies that acknowledge interactions between pickers and offline customers, we formalize a new problem called Dynamic In-store Picker Routing Problem (diPRP). This problem considers a picker that tries to pick online orders (seeking) while minimizing customer encounters (hiding) – preserving the offline customer experience. We model the problem as a Markov Decision Process (MDP) and solve it using a hybrid solution approach comprising mathematical programming and reinforcement learning components. Computational experiments on synthetic instances suggest that the algorithm converges to efficient policies. We apply our solution approach in the context of a large European retailer to assess the proposed policies regarding the number of orders picked and customers encountered. The learned policies are also tested in six different retail settings, demonstrating the flexibility of the proposed approach. Our work suggests that retailers should be able to scale the in-store picking of online orders without jeopardizing the experience of offline customers. The policies learned using the proposed solution approach reduced the number of customer encounters by up to 50%, compared to policies solely focused on picking orders. Thus, to pursue omnichannel strategies that adequately trade-off operational efficiency and customer experience, retailers cannot rely on actual simplistic picking strategies, such as choosing the shortest possible route.

Time Window Oriented IoT Vehicle Pathway Study for the Dynamically Changing Needs of E-Commerce Customers

The main dynamic truck routing problem also presents a significant difficulty in the logistics sector, which is an unavoidable development trend of the contemporary technological changing society. A dynamic vehicle routing problem with time window model is suggested by the study in order to establish an effective and low-energy dynamic response method. The fundamental concept is to disrupt the conventional strategy of static dynamic consumers responding in time slots by dividing the dynamic time window into a static time window with several time slice intervals. The study makes use of cutting-edge ideas including dynamic attitude, before-and-after time slicing, and continuous optimisation while proposing a new method for model solution to optimise dynamic vehicle route issues effectively and affordably. The study employs the Solomon optimisation dataset and runs simulation studies on the Java platform to confirm its efficacy. The experimental findings demonstrated that the optimisation technique employed in the study reduced the cost of travelling by 83.8 miles while also considerably increasing the average vehicle utilisation by 3.6%. Because driving distance cost and vehicle number cost are typically positively connected with dynamic attitude, the study employs solutions that can increase dynamic response efficiency and save money. As a result, their robustness is higher.

A RMSA algorithm for semi-filterless optical networks based on minimum resource consumption and spectrum fragmentation

Semi-filterless optical networks (semi-FONs), a novel passive wide area network (WAN) solution, have attracted considerable attention. This paper investigates the dynamic routing, modulation and spectrum assignment (RMSA) problem in semi-FONs, where requests may propagate on unnecessary links due to the broadcast characteristics of semi-FONs, resulting in wastage of spectrum resources. Moreover, the dynamically setup and torn down of requests can lead to significant spectrum fragments. To address these issues, this paper studies semi-FONs based on two metrics: resource consumption M and fragmentation degree N. Resource consumption M refers to reducing spectrum wastage caused by broadcasting and establishing requests using minimal resources, while fragmentation degree N refers to the situation where the remaining available resources after establishing a request are more concentrated. Based on these metrics, we propose the Minimum Resource Consumption and Fragmentation Degree (Min-RCF) algorithm. We conduct simulation experiments on the German-Net and China’s Henan-Net network topologies, and the results demonstrate that the proposed Min-RCF algorithm achieves lower blocking probability and spectrum waste rate than the compared algorithms.

Designing a multi-period dynamic electric vehicle production-routing problem in a supply chain considering energy consumption

The coordinated decision-making approach for considering sequential activities of the supply chain results in additional benefits by optimizing production, inventory, and distribution operations. Accordingly, this paper proposes a mixed integer linear mathematical model to optimize a multi-period production routing problem utilizing electric vehicles. The proposed model optimizes the total cost associated with fixed and variable production, holding inventory, and routing, including the fixed cost of utilizing electric vehicles and travel time. However, mileage limitation is one of the main restrictions of utilizing electric vehicles in performing deliveries which is strongly affected by the consumed energy. Although optimization of the routes of vehicles can facilitate using them, considering the variation of travel speed network links during different times of the day because of traffic conditions can obviously affect the required energy to perform the assigned deliveries. To the best of our knowledge, this paper is the first to study simultaneous multi-period dynamic production routing problems using a set of heterogenous electric vehicles whose travel time of links can vary by dividing each production period into several hourly time intervals to capture different traffic conditions. Finally, a series of capable and hybrid metaheuristic algorithms are designed and implemented to solve this problem in a real-case dimension, and all proposed algorithms are compared.

Tactical planning for dynamic technician routing and scheduling problems

We consider the tactical planning aspect of a dynamic technician routing and scheduling problem with a time horizon of several days. In this problem study, some tasks are known beforehand, while others arrive dynamically. The tactical planning is to schedule the known tasks, such that we minimize the overall driving distance while ensuring short service times for the dynamic tasks. Without tactical planning, the known tasks would be spread throughout the whole area as they are scheduled based on a first-come-first-serve principle. In this study, we partition the area into disjoint slices covering the full plane and then assign the slices to individual work days. The partitioning of the area will be constructed with a balanced sweep algorithm using two approaches: The first is based on a parallel sweep-line following a given angle, while the second is based on a rotating sweep-line around a given origin. To guide the partitioning, we aim to minimize the degree of dynamism, which in our case is defined by the number of dynamic tasks, the distance from known tasks to dynamic tasks, and the tightness of the time windows. The tactical planning is evaluated using real-life and synthetic clustered data, where some of the tasks are known beforehand, and some are dynamic. We benchmark the various partitioning approaches against the default strategy of serving all known tasks on a first-come-first-serve basis. Computational results are reported showing around a 10% reduction in driving distance when using tactical planning. Furthermore, we show that up to 70% of the technicians can have a non-dynamic work day, without a significant increase in driving distance.

Dynamic container drayage booking and routing decision support approach for E-commerce platforms

E-commerce drayage platforms are relatively new to the container drayage industry. Their revenue management poses an enormous logistical challenge owing to the unpredictable demand coupled with complex operations. In this paper, we propose a novel dynamic container drayage booking and routing problem with uncertain demand that incorporates important features of this emerging business model from the perspective of an e-commerce drayage platform. We formulate a multi-stage stochastic programming model for problem followed by developing an effective two-phase heuristic method based on a multi-scenario approach to derive both service booking and fleet routing solutions. In the two-phase heuristic, a container drayage service booking sub-problem is solved at the first phase to determine whether each request should be accepted for service or rejected, as well as the time period for serving the accepted requests, while a fleet routing sub-problem is tackled at the second phase to generate work schedules for the tractors and trailers to transport the containers. We conduct comprehensive computational experiments to demonstrate the quality and value of the proposed two-phase heuristic, and reveal insights into the practical operations management of e-commerce drayage platforms.

Prepositioning can improve the performance of a dynamic stochastic on-demand public bus system

This paper presents the first application of prepositioning in the context of the dynamic stochastic on-demand bus routing problem (DODBRP). The DODBRP is a large-scale dial-a-ride problem that involves bus station assignment and aims to minimize the total user ride time (URT) by simultaneously assigning passengers to alternative stations and determining optimal bus routes.In the DODBRP, transportation requests are introduced dynamically, and buses are dispatched to stations with known requests. This paper investigates the concept of prepositioning, which involves sending buses not only to currently known requests but also to requests that are likely to appear in the future, based on a given probability.To solve this dynamic and stochastic ODBRP, the paper proposes a heuristic algorithm based on variable neighborhood search (VNS). The algorithm considers multiple scenarios to represent different realizations of the stochastic requests.Experimental results demonstrate the superiority of the prepositioning approach over the DODBRP across various levels of forecast accuracy, lengths of time bucket, and probabilities of realization. Furthermore, the paper shows that removing empty stations as a recourse action can further enhance solution quality. Additionally, in situations with low prediction accuracy, increasing the number of scenarios can lead to improved solutions. Finally, a combination of prepositioning, empty station removal, and the insertion of dynamic requests proves to be effective.Overall, the findings of this paper provide valuable insights into the application of prepositioning in the dynamic stochastic on-demand bus routing problem, highlighting its potential for addressing real-world transportation challenges.

Dynamic parcel pick-up routing problem with prioritized customers and constrained capacity via lower-bound-based rollout approach

In this study, we address a parcel pick-up routing problem that involves prioritized customers and constrained capacity, in the context of dynamic vehicle routing. The number and locations of potential customers are known in advance, but the sizes of their parcels are stochastic and unknown until their requests reveal. To tackle this problem, we formulate it as a Markov decision process (MDP), where dynamic requests are handled periodically. Specifically, in each stage, if the total size of the dynamic customers’ parcels that have accumulated fits within the vehicle’s remaining capacity, all of the requests are accepted. Otherwise, requests are accepted according to customers’ priorities. Our objective is to minimize the total expected waiting time of customers. However, the multi-dimensional MDP state causes the curse of dimensionality, making the problem challenging to solve. Therefore, we propose a rollout approach to solve the problem. First, unlike traditional rollout-related methods, we derive a lower bound of the post-decision state’s value function and use it to develop a base policy. Second, we apply a post-decision rollout algorithm to determine an online rollout policy that theoretically satisfies the rolloutimproving property. In our numerical study, we compare our rollout policy with a general value function approximation-based rollout policy and a model-independent greedy method. Our results show that our rollout policy outperforms both of them with a significant advantage, while the latter two methods have ups and downs on both sides.

Learning Heuristics With Different Representations for Stochastic Routing.

Uncertainty is ubiquitous in real-world routing applications. The automated design of the routing policy by hyperheuristic methods is an effective technique to handle the uncertainty and to achieve online routing for dynamic or stochastic routing problems. Currently, the tree representation routing policy evolved by genetic programming is commonly adopted because of the remarkable flexibility. However, numeric representations have never been used. Considering the practicability of the numeric representations and the capability of the numeric optimization methods, in this article, we investigate two numeric representations on a representative stochastic routing problem and uncertain capacitated arc routing problem. Specifically, a linear representation and an artificial neural-network (ANN) representation are implemented and compared with the tree representation to reveal the potential of the numeric representations and the characteristics of their optimization. Experimental results show that the tree representation is the best choice, but on a majority of the test instances, the numeric representations, especially the ANN representation, can provide competitive performance. Further analyses also show that training a good ANN representation policy requires more training data than the tree representation. Finally, a guideline of representation selection is given.