Routing Policies Research Articles

Emergency Care Efficiency vs. Quality: Uncovering Hidden Consequences of Fast-Track Routing Decisions

Problem definition: This work aims to examine the role of emergency department (ED) operational status related to congestion in fast-track (FT) routing decisions and the subsequent effects on patient outcomes. Methodology/results: In this paper, we utilize a two-year data set from two hospital EDs in Alberta, Canada, and adopt an instrumental variable approach to examine the effects of FT routing decisions on patient outcomes. Based on the empirical findings, we utilize a data-calibrated simulation to compare the performance of different routing policies. First, our study reveals that FT routing decisions are not purely clinically driven, and ED operational status is also associated with FT routing decisions. Second, being routed to FT can improve ED efficiency by reducing the average length of stay and left without being seen rates. However, this efficiency improvement comes at the cost of potential quality decline. In particular, being routed to the FT leads to an 8.2% increase in the 48-hour revisit rate for the high-complexity group and a 2.3% increase for the medium-complexity group. Third, we delve into the mechanisms behind observed patient outcomes and find that physicians in the FT area may prioritize expediting patient flow by simplifying patient diagnosis and treatment procedures. Consequently, the quality of care may be compromised for high- and medium-complexity patients. Finally, our simulation findings highlight the importance of selecting the “right” patients to be routed to the FT unit. To this end, the complexity-based classification method and dynamic routing policies emerge as promising avenues. Managerial implications: Our findings call for immediate attention from healthcare practitioners to carefully balance the trade-off between emergency care efficiency and quality, emphasizing the necessity of selecting the right patients for routing. Funding: This study is partially supported by the Hong Kong Research Grants Council [Grants GRF 11508921 and CRF C7162-20G]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2022.0440 .

Lightweight Flow‐Based Policy Enforcement for SDN‐Based Multi‐Domain Communication

ABSTRACTAlthough software‐defined networking (SDN) is commonly employed for intra‐domain communication, inter‐domain communication still heavily relies on conventional routing methods, specifically BGP‐based routers. The BGP router plays a crucial role in managing control and data planes, but this traditional approach hinders the exploitation of SDN advantages. Previous studies demonstrated the use of BGP for inter‐domain and end‐to‐end communication. This paper advocates for the adoption of a fully SDN‐based data plane packet switching strategy through the introduction of LPEES, a lightweight policy framework tailored for SDN‐based inter‐domain communication. LPEES strategically confines BGP's functionality to the control plane, preserving SDN benefits. Evaluation results confirm the effectiveness of LPEES compared to the BGP routing approach, as measured by throughput and various network quality of service (QoS) metrics. Additionally, LPEES streamlines inter‐domain communication by utilizing a trust‐based routing policy approach that can establish trust between communicating domains. The presented solution's main advantage is that it loosens the burden on the administrator by requiring less human interference to check the inter‐domain communication security and privacy. Our evaluations show LPEES outperform the BGP‐based in terms of throughput as LPEES achieves a 27 Gbps versus 22 Gbps in the traditional approach. Based on our experiments, LPEES also enhances the communication delay by an average of 17% compared to the traditional BGP‐based approach.

Optimal resilient fleet route policy in shipping industry: Two-stage model for minimizing disruption-related effect

Optimal resilient fleet route policy in shipping industry: Two-stage model for minimizing disruption-related effect

Evolving routing policies for electric vehicles by means of genetic programming

AbstractIn recent years, the growing interest in environmental sustainability has led to Electric Vehicle Routing Problems (EVRPs) attracting more and more attention. EVRPs involve the use of electric vehicles, which have additional constraints, such as range and recharging time, compared to conventional Vehicle Routing Problems (VRPs). The complexity and dynamic nature of solving VRPs often lead to the introduction of Routing Policies (RPs), simple heuristics that incrementally build routes. However, manually designing efficient RPs proves to be a challenging and time-consuming task. Therefore, there is a pressing need to explore the application of hyper-heuristics, in particular Genetic Programming (GP), to automatically generate new RPs. Since this method has not yet been investigated in the literature in the context of EVRPs, this study explores the applicability of GP to automatically generate new RPs for EVRP. To this end, three RP variants (serial, semiparallel, and parallel) are introduced in this study, along with a set of domain-specific terminal nodes to optimise three criteria: the number of vehicles, energy consumption, and total tardiness. The experimental analysis shows that the serial variant performs best in terms of energy consumption and number of vehicles, while the parallel variant is most effective in minimising the total tardiness. A comprehensive analysis of the proposed method is conducted to determine its convergence properties and the impact of the proposed terminal nodes on performance and to describe several generated RPs. The results show that the automatically generated RPs perform commendably compared to traditional methods such as metaheuristics and exact methods, which usually require significantly more runtime. More specifically, depending on the scenario in which they are used, the generated RPs achieve results that are about 20%-37% worse compared to the best known results for the number of vehicles in almost negligible time, in just some milliseconds.

Asymptotically optimal routing of a many-server parallel queueing system with long-run average criterion

This paper considers a parallel queueing system with multiple stations, each of which contains many statistically identical servers and has a dedicated queue. Upon each customer arrival, the system manager must decide to which station the customer should be routed, with the objective of minimizing the system’s long-run average delay cost. One feature of this paper is that a customer’s delay cost depends not only on his/her delay, but also on the routed station. Considering this heterogeneity across stations, we propose a routing policy, which can be regarded as an extension of the MED–FSF policy. Under this policy, any arriving customer will be routed to: (i) the station with the minimum value, which depends on the station’s expected delay and the station index when servers in all stations are fully occupied; or otherwise (ii) the station with a fastest idle server. Using asymptotic analysis, we derive diffusion limits of queue-length processes and their stationary distributions under the proposed policy in the Halfin–Whitt regime. Combined with an asymptotic lower bound result for the long-run average delay cost, we show that the proposed routing policy is asymptotically optimal under the considered objective. Finally, we provide numerical experiments to validate the accuracy of our diffusion approximation, and we compare the performance metrics under the proposed policy with those under other commonly used routing policies.

Do CBDCs promote financial inclusion and strengthen the monetary regulations?

Digital currencies are impacting the financial lives of people and countries around the world; particularly developing countries see the retail Central Bank Digital Currencies (CBDCs) as an opportunity to increase financial inclusion and introduce new monetary policy implementation mechanisms. The decision to launch a digital currency mainly depends upon the goals and objectives being set by the Central Bank of any respective country. Digital currencies help to implement monetary policy more effectively to achieve the financial system stability. Kazakhstan is the leading economy in the Commonwealth of Independent States region; the digital payments market is changing rapidly in Kazakhstan, being driven by fintech innovations and public investments in payment technologies. In this article, we thoroughly discussed how launching of “Digital Tenge” is promoting financial inclusion and strengthening the monetary regulations across Kazakhstan. We took the case of Kazakhstan because it’s the first country from Central Asia which took the initiative of launching digital currency, and the National Bank of Kazakhstan has already announced that implementation of “Digital Tenge” would be completed in Kazakhstan by the end of 2025. The implementation of a “Digital Tenge” as a CBDC in Kazakhstan would expand the range of financial services being available to the nation. Having direct authority over the digital currency will empower the National Bank of Kazakhstan to adopt more efficient monetary policy, exerting influence over interest rates, money supply, and general economic stability. Furthermore, by providing digital financial services to marginalized communities, the digital Tenge would also help to solve the difficulties of financial inclusion in Kazakhstan. The article provides the possible policy routes for the governments of developing countries and the financial regulators about adopting CBDCs for promoting financial inclusion and to strengthen the monetary regulations.

Explaining Genetic Programming-Evolved Routing Policies for Uncertain Capacitated Arc Routing Problems

Genetic programming has been successfully used to evolve routing policies that can make real-time routing decisions for uncertain arc routing problems. Although the evolved routing policies are highly effective, they are typically very large and complex, and hard to be understood and trusted by real users. Existing studies have attempted to improve the interpretability by developing new genetic programming approaches to evolve both effective and interpretable (e.g., with smaller program size) routing policies. However, they still have limitations due to the trade-off between effectiveness and interpretability. To address this issue, we propose a new post-hoc explanation approach to explaining the effective but complex routing policies evolved by genetic programming. The new approach includes a local ranking explanation and a global explanation module. The local ranking explanation uses particle swarm optimisation to learn an interpretable linear model that accurately explains the local behaviour of the routing policy for each decision situation. Then, the global explanation module uses a clustering technique to summarise the local explanations into a global explanation. The experimental results and case studies on the benchmark datasets show that the proposed method can obtain accurate and understandable explanations of the routing policies evolved for uncertain arc routing problems. Our explanation approach is not restricted to uncertain arc routing, but has a great potential to be generalised to other optimisation and machine learning problems such as learning classifier systems and reinforcement learning.

SDNRoute: Proactive routing optimization in Software Defined Networks

Despite opening attractive perspectives, the concept of Software Defined Networking raises doubts about the performance and practical feasibility. To contradict these concerns, we propose a deployment-ready system aimed at proactive and periodic optimization of flow paths. The modular system consists of modules responsible for traffic prediction, static optimization, measurements, flow management, and validation of optimization results. To make the system efficient, we resolved several scientific issues and proposed novel and valuable solutions, for example methods for efficient proactive flow management and periodic re-optimization of routing policies. Simultaneously, to make the system production-ready and create a reliable research environment, we provide solutions to several technical obstacles.We validate the proposed system with three network topologies, each with three load levels, following real-life traffic models. We consider Equal-Cost Multi-Path Routing as a baseline. The results indicate that the system allows network operators to handle more traffic (packet loss reduced by up to 30%), improve quality of service (less congested links resulted in even 2.5 times lower latency), and reduce operational expenses (energy consumption lowered by up to 10%).

TITE: A transformer-based deep reinforcement learning approach for traffic engineering in hybrid SDN with dynamic traffic

Hybrid Software Defined Networks (Hybrid SDNs), with a partial upgrade of legacy routers to SDN switches in traditional distributed networks, currently stand as a prevailing network architecture. Traffic Engineering (TE) in hybrid SDN requires the efficient and timely acquisition of a routing policy to adapt to dynamically changing traffic demands, which has recently become a hot topic. Ignoring the hidden relations of consecutive states and the compact representation of the network environment, previous Deep Reinforcement Learning (DRL)-based studies suffer from the convergence problem by only establishing the direct relationship between individual Traffic Matrix (TM) and routing policy. Therefore, to enhance TE performance in hybrid SDNs under dynamically changing traffic demands, we propose to integrate the Transformer model with DRL to establish the relationship between consecutive states and routing policies. The temporal characteristic among consecutive states can effectively assist DRL in solving the convergence problem. To obtain a compact and accurate description of the network environment, we propose to jointly consider TM, routing action, and reward in designing the state of the network environment. To better capture the temporal relations among consecutive states of the network environment, we design a multi-feature embedding module and achieve positional encodings in the Transformer model. The extensive experiments demonstrate that once the convergence problem is solved, the proposed Transformer-based DRL method can efficiently generate routing policies that adapt well to dynamic network traffic.

GROM: A generalized routing optimization method with graph neural network and deep reinforcement learning

Routing optimization, as a significant part of Traffic Engineering (TE), plays an important role in balancing network traffic and improving quality of service. With the application of Machine Learning (ML) in various fields, many neural network-based routing optimization solutions have been proposed. However, most existing ML-based methods need to retrain the model when confronted with a network unseen during training, which incurs significant time overhead and response delay. To improve the generalization ability of the routing model, in this paper, we innovatively propose a routing optimization method GROM which combines Deep Reinforcement Learning (DRL) and Graph Neural Networks (GNN), to directly generate routing policies under different and unseen network topologies without retraining. Specifically, for handling different network topologies, we transform the traffic-splitting ratio into element-level output of GNN model. To make the DRL agent easier to converge and well generalize to unseen topologies, we discretize the huge continuous traffic-splitting action space. Extensive simulation results on five real-world network topologies demonstrate that GROM can rapidly generate routing policies under different network topologies and has superior generalization ability.

The Co-Production of Service: Modeling Services in Contact Centers Using Hawkes Processes

In customer support contact centers, every service interaction involves a messaging dialogue between a customer and an agent; together, they exchange information, solve problems, and collectively co-produce the service. Because the service progression is shaped by the history of conversation thus far, we propose a bivariate marked Hawkes process cluster model of the customer-agent interaction. To evaluate our stochastic model of service, we apply it to an industry contact center data set containing nearly 5 million messages. Through both a novel residual analysis comparison and several Monte Carlo goodness-of-fit tests, we show that the Hawkes cluster model indeed captures dynamics at the heart of the service and surpasses classic models that do not incorporate the service history. Furthermore, in an entirely data-driven simulation, we demonstrate how this history-dependent model can be leveraged operationally to inform a prediction-based routing policy. We show that widely used and well-studied customer routing policies can be outperformed with simple modifications according to the Hawkes model. Through analysis of a stylized model proposed in the contact center literature, we prove that service heterogeneity can cause this underperformance and, moreover, that such heterogeneity will occur if service closures are not carefully managed. This paper was accepted by Elena Katok, operations management. Funding: The authors are grateful for the generous support of this work by the National Science Foundation Division of Graduate Education [Grant DGE-1650441] (A. Daw), the Israel Science Foundation [Grant 336/19] (G. B. Yom-Tov), and the United States-Israel Binational Science Foundation [Grant 2022095] (A. Daw, G. B. Yom-Tov). Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2021.04060 .

RON‐based cross‐chain routing optimization strategy in metaverse

AbstractMetaverse is a new ecology that integrates the digital world with the physical world, generates a mirror image of the real world based on digital twin technology, and guarantees the fairness of the virtual world through the trusted mechanism of blockchain. Cross‐chain communication technology is the key to realizing data circulation and collaborative processing between blockchains, which provides the communication foundation for the massive connection of blockchains in the metaverse. The current cross‐chain communication mode is dominated by direct‐connect routing, leading to network congestion and high propagation delay once the direct‐connect link fails and cannot be recovered quickly. To optimize direct‐connect routing, this paper proposed a cross‐chain routing optimization strategy based on RON (Resilient Overlay Network), that is, Cross‐Chain_RON, which firstly applies RON to reconstruct the direct‐connect routing model, and then selects the optimal link through the shortest‐path algorithm and policy routing, and combines with the RON performance database to improve the data transmission efficiency. Finally, the two communication modes were simulated by simulation tools. The experiments show that Cross‐Chain_RON outperforms the direct‐connect routing in terms of latency, rate of successful data transmission, and throughput in poor network environments, but at the expense of a certain amount of system overhead.

Adaptive stochastic lookahead policies for dynamic multi-period purchasing and inventory routing

We explore a problem faced by agri-food e-commerce platforms in purchasing different, perishable products and collecting them from multiple producers and delivering them to a single warehouse, aiming to maintain adequate inventory levels to meet current and future customer demand, while avoiding waste. Customer demand and suppliers’ purchase prices and supply volumes are uncertain and revealed on a periodical basis. Every period, purchasing, inventory, and routing decisions are made to satisfy demand and to build inventory for future periods. For effective decisions integrating all three decision components and anticipating future developments, we propose a stochastic lookahead method that, in every period, samples future scenarios for demand, supply volumes, and prices. It then solves a two-stage stochastic program to obtain the decision for the current period. To make this approach computationally tractable, we reduce the routing decision in the two-stage program and use an approximate routing cost instead. Given the reduced decision, we then create the final decision via a conventional routing heuristic. We learn the routing cost approximation adaptively via repeated training simulations. In comprehensive experiments, we show that all three components, stochastic lookahead, routing cost approximation, and adaptive learning, are very effective individually, but especially in combination. We also provide a comprehensive analysis of the problem parameters and obtain valuable insights in problem and methodology.

Markov game for CV joint adaptive routing in stochastic traffic networks: A scalable learning approach

This study proposes a learning-based approach to tackle the challenge of joint adaptive routing in stochastic traffic networks with Connected Vehicles (CVs). We introduce a Markov Routing Game (MRG) to model the adaptive routing behavior of all vehicles in such networks, thereby incorporating both competitive route choices and real-time decision-making. We establish the existence of the Nash policy (i.e., optimal joint adaptive routing policy) within the MRG that enables vehicles to adapt optimally to real-time traffic conditions online through efficient communication. To enhance scalability, we innovate with a homogeneity-based mean-field approximation method and, based on that, further develop the Homogeneity-based Mean-Field Deep Reinforcement Learning (HMF-DRL) algorithm to learn the Nash policy within the MRG. Through numerical experiments on the Nguyen–Dupuis network, we demonstrate our algorithm’s ability to efficiently converge and learn the joint adaptive routing policy that significantly enhances traffic network efficiency. Furthermore, our study provides insights into the effects of travel demand, penetration of CVs, and levels of uncertainty on the performance of the joint adaptive routing policy. This paper presents a significant step towards improving network efficiency and reducing the travel time for a majority of vehicles amid uncertain traffic conditions.

Performance Analysis of Multi-Tote Storage and Retrieval Autonomous Mobile Robot Systems

Multi-tote storage and retrieval (MTSR) autonomous mobile robots can carry multiple product totes, store and retrieve them from different shelf rack tiers, and transport them to a workstation where the products are picked to fulfill customer orders. In each robot trip, totes retrieved during the previous trip must be stored. This leads to a mixed storage and retrieval route. We analyze this mixed storage and retrieval route problem and derive the optimal travel route for a multiblock warehouse by a layered graph algorithm, based on storage first-retrieval second and mixed storage and retrieval policies. We also propose an effective heuristic routing policy, the closest retrieval (CR) sequence policy, based on a local shortest path. Numerical results show that the CR policy leads to shorter travel times than the well-known S-shape policy, whereas the gap with the optimal mixed storage and retrieval policy in practical scenarios is small. Based on the CR policy, we model the stochastic behavior of the system using a semiopen queuing network (SOQN). This model can accurately estimate average tote throughput time and system throughput capacity as a function of the number of robots in the system. We use the SOQN and corresponding closed queuing network models to optimize the total annual cost as a function of the warehouse shape, the number of robots, and tote buffer positions on the robots for a given average tote throughput time and throughput capacity. Compared with robots that retrieve a single tote per trip, an MTSR system with at least five buffer positions can achieve lower operational costs while meeting given average tote throughput time and tote throughput capacity constraints. Funding: This work was supported by National Natural Science Foundation of China [Grant 72372088] and the Shenzhen Science and Technology Program [Grant GJHZ20220913143003006]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/trsc.2023.0397 .

The single picker routing problem with scattered storage: modeling and evaluation of routing and storage policies

Despite ongoing automation efforts, most warehouses are still manually operated using a person-to-parts collection strategy. This process of collecting items of customer orders from different storage locations accounts for the majority of the operating costs of the warehouse. Hence, optimizing picker routes is an important instrument to reduce labor costs. We examine the scattered-storage variant of the single picker routing problem in a one-block parallel-aisle warehouse. With scattered storage, an article can be stored at several storage locations within the warehouse, whereas with classic storage, each article has a unique storage location. We use our recently published network-flow model with covering constraints that is based on an extension of the state space of the dynamic-programming formulation by Ratliff and Rosenthal. With modifications in the state graph, this model serves for both exact and all established heuristic routing methods for picker routing. The latter include traversal, return, largest gap, midpoint, and composite. We show that these routing policies can also be implemented through adaptations in the state space. Extensive computational studies highlight a comparison of the different routing and storage policies (in particular class-based storage policies) in the scattered storage context. Analyses demonstrate which combinations of policies are advantageous for the given warehouse layout. For class-based storage policies, we emphasize how the scattering of articles of different classes should be performed: scattering of C-articles is advantageous with reductions of up to 25%. In contrast, when articles are uniformly distributed, A-articles should be scattered.

Energy-Efficient Virtual Network Embedding: A Deep Reinforcement Learning Approach Based on Graph Convolutional Networks

Network virtualization (NV) technology is the cornerstone of modern network architectures, offering significant advantages in resource utilization, flexibility, security, and streamlined management. By enabling the deployment of multiple virtual network requests (VNRs) within a single base network through virtual network embedding (VNE), NV technology can substantially reduce the operational costs and energy consumption. However, the existing algorithms for energy-efficient VNE have limitations, including manual tuning for heuristic routing policies, inefficient feature extraction in traditional intelligent algorithms, and a lack of consideration of periodic traffic fluctuations. To address these limitations, this paper introduces a novel approach that leverages deep reinforcement learning (DRL) to enhance the efficiency of traditional methods. We employ graph convolutional networks (GCNs) for feature extraction, capturing the nuances of network graph structures, and integrate periodic traffic fluctuations as a key constraint in our model. This allows for the predictive embedding of VNRs that is both energy-efficient and responsive to dynamic network conditions. Our research aims to develop an energy-efficient VNE algorithm that dynamically adapts to network traffic patterns, thereby optimizing resource allocation and reducing energy consumption. Extensive simulation experiments demonstrate that our proposed algorithm achieves an average reduction of 22.4% in energy consumption and 41.0% in active substrate nodes, along with a 23.4% improvement in the acceptance rate compared to other algorithms.

A Study Comparing Waiting Times in Global and Local Queuing Systems with Heterogeneous Workers

A virtual marketplace or service-providing system must ensure minimal task response times. Varying working rates among the human workers in the system can lead to longer delays for certain tasks. The waiting time in the queue is crucially affected by the queueing architecture used in the system, whether global or local. Studies generally favor global queue systems over local ones, assuming similar processing rates. However, system behavior changes when workers are heterogeneous. In this research, we used simulation to compare the waiting times of tasks assigned to three categories of processing rates in both architectures and with various routing policies in local queues. We found that when using random tie-breaking, there was a correlation between waiting time duration and the proportion of tie-breaking events. Performance is improved when controlling these events using scheduling awareness of the workers’ processing rates. The global queue outperforms local queues when the workers are homogeneous. However, the push mechanisms that control the assignment processes and heterogeneity-aware algorithms improve local queue system waiting times and load balance. It is better than global queues when tasks are assigned to medium and fast workers, but it also enables specific slow workers’ assignments.

Ship Network Traffic Engineering Based on Reinforcement Learning

This research addresses multiple challenges faced by ship networks, including limited bandwidth, unstable network connections, high latency, and command priority. To solve these problems, we used reinforcement learning-based methods to simulate traffic engineering in ship networks. We focused on three aspects—traffic balance, instruction priority, and complex network structure—to evaluate reinforcement learning performance in these scenarios. Performance: We developed a reinforcement learning framework for ship network traffic engineering that treats the routing policy as the state and the network state as the environment. The agent generates routing changes and uses actions to optimize traffic services. The experimental results show that reinforcement learning optimizes network traffic balance, reasonably arranges instruction priorities, and copes with complex network structures, greatly improving the network’s quality of service (QoS). Through an in-depth analysis of the experimental data, we noticed that network consumption was reduced by 9.1% under reinforcement learning. Reinforcement learning effectively implemented priority routing of high-priority instructions while reducing the occupancy rate of the edge with the highest occupancy rate in the network by 18.53%.

Cache Update and Delivery of Dynamic Contents: A Stochastic Game Approach

In this paper, we propose a cache update policy for wireless networks considering dynamic popularity for file requests. Our network scenario is a wireless network comprised of some cache-equipped access points (APs) which are deployed densely in an area and have connections with core network servers. We model the dynamics of file requests as Markov-modulated Poisson processes. Considering the congestion-dependent delay of APs' services and their overlapping coverage areas, we model the cache service for the cache-missed requests, as a stochastic game in which a Vickrey–Clarke–Groves (VCG) mechanism is exploited at each stage of the game as our proposed file routing policy, and the APs decide independently how to update their caches regarding the routed files. To this end, the APs' utilities are defined based on long-term average file delivery delay including the queueing delay in APs. By finding the Nash equilibrium of the game, we propose policies for both delivery and cache update of dynamic contents. Finally, by comparing numerical results of our proposed scheme with some conventional caching schemes, we show significant improvements in network performance in terms of file delivery delay in different conditions.