Scheduling Policy Research Articles

A novel multipath QUIC protocol with minimized flow complete time for internet content distribution

The rapid growth of network services and applications has led to an exponential increase in data flows on the internet. Given the dynamic nature of data traffic in the realm of internet content distribution, traditional TCP/IP network systems often struggle to guarantee reliable network resource utilization and management. The recent advancement of the Quick UDP Internet Connect (QUIC) protocol equips media transfer applications with essential features, including structured flowcontrolled streams, quick connection establishment, and seamless network path migration. These features are vital for ensuring the efficiency and reliability of network performance and resource utilization, especially when network hosts transmit data flows over end-to-end paths between two endpoints. QUIC greatly improves media transfer performance by reducing both connection setup time and transmission latency. However, it is still constrained by the limitations of single-path bandwidth capacity and its variability. To address this inherent limitation, recent research has delved into the concept of multipath QUIC, which utilizes multiple network paths to transmit data flows concurrently. The benefits of multipath QUIC are twofold: it boosts the overall bandwidth capacity and mitigates flow congestion issues that might plague individual paths. However, many previous studies have depended on basic scheduling policies, like round-robin or shortest-time-first, to distribute data transmission across multiple paths. These policies often overlook the subtle characteristics of network paths, leading to increased link congestion and transmission costs. In this paper, we introduce a novel multipath QUIC strategy aimed at minimizing flow completion time while taking into account both path delay and packet loss rate. Experimental results demonstrate the superiority of our proposed method compared to standard QUIC, Lowest-RTT-First (LRF) QUIC, and Pluginized QUIC schemes. The relative performance underscores the efficacy of our design in achieving efficient and reliable data transfer in real-world scenarios using the Mininet simulator.

Structure-Enhanced DRL for Optimal Transmission Scheduling

Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, we focus on the transmission scheduling problem of a remote estimation system. First, we derive some structural properties of the optimal sensor scheduling policy over fading channels. Then, building on these theoretical guidelines, we develop a structure-enhanced deep reinforcement learning (DRL) framework for optimal scheduling of the system to achieve the minimum overall estimation mean-square error (MSE). In particular, we propose a structure-enhanced action selection method, which tends to select actions that obey the policy structure. This explores the action space more effectively and enhances the learning efficiency of DRL agents. Furthermore, we introduce a structure-enhanced loss function to add penalties to actions that do not follow the policy structure. The new loss function guides the DRL to converge to the optimal policy structure quickly. Our numerical experiments illustrate that the proposed structure-enhanced DRL algorithms can save the training time by 50% and reduce the remote estimation MSE by 10% to 25%, when compared to benchmark DRL algorithms. In addition, we show that the derived structural properties exist in a wide range of dynamic scheduling problems that go beyond remote state estimation.

An Adaptive Dynamic Scheduling Policy for the Integrated Optimization Problem in Automated Transshipment Hubs

This study considers the multi-period dynamic integrated optimization problem in automated transshipment hubs, which includes berth allocation, quay crane assignment, and yard assignment problems. The planner needs to determine the berth and storage schedule in real-time. In the previous literature, the scholars always assume that all of the required information, especially the arrival time and operation time of vessels, is accessible prior to making any design decisions, and formulate the problem into an integrated model. However, the information of vessels is updated dynamically and changed frequently. Therefore, we consider a problem in a dynamic setting that decisions are made in each period and the scenario at each period depends on the decisions in previous periods. We formulate the problem into a multi-objective model, which aims to maximize total revenue, saved time deviation, saved transportation distance, and service quality. An efficient adaptive dynamic scheduling policy is developed, which adaptively balances the trade-offs between multiple objectives. Through numerical experiments, we demonstrate that our approach presents solutions with a delicate balance between multiple objectives and brings value to the automated container terminals compared to benchmark policies. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Automated technologies have brought great opportunities for container terminals. While the construction of an automated container terminal requires enormous investment. Utilizing the resources of container terminals reasonably and efficiently can improve the operation efficiency and economic performance of automated container terminals. In practice, the scheduling of resources in container terminals always encounters uncertain factors. Therefore, this study proposes a dynamic integrated optimization policy for resources in automated container terminals to balance the trade-offs between multiple objectives given any pre-determined target in a dynamic environment.

Nonsequential Appointment Scheduling With a Random Number of Requests

This article studies an appointment scheduling problem where a service provider dynamically receives appointment requests from a random number of customers. By leveraging the randomness of the number of potential customers, we develop a nonsequential appointment scheduling policy as an alternative to the conventional first-come-first-served (FCFS) policy. This allows for more flexibility in managing appointment scheduling. To calculate the optimal policy, we develop a branch-and-bound algorithm in which the lower bound is estimated using multiple approaches, such as optimality conditions, dynamic programming for calculating FCFS policy, and the shortest path reformulation. Through numerical studies, we observe that nonsequential appointment scheduling is particularly advantageous in systems characterized by highly fluctuating customer numbers or low congestion. In such cases, leaving gaps between appointments for potential future arrivals proves to be a more appropriate strategy. We also evaluate the performance of heuristics proposed in prior literature and provide insights into situations where these heuristics can be effectively applied.

Digital twins, the journey of an operational weather prediction system into the heart of Destination Earth

Moving a world leading numerical weather prediction system of the European Center for Medium-Range Weather Forecasts (ECMWF), that runs on a dedicated, bespoke, high performance computing cluster and supporting infrastructure, into the heart of the digital twin for climate change adaptation and extreme weather events, developed in the Destination Earth (DestinE) initiative of the European Commission, has been a challenging and exciting journey. In this paper we describe this journey with a focus on those aspects required to leverage the pre-exascale EuroHPC systems that have been made available to DestinE to run its computational representation [1]. EuroHPC systems can be effectively used for DestinE and are in fact key assets to deliver the computational power required for Earth system digital twins at global km-scale resolution. Each of these systems is operated by a national hosting entity that implements its own procedures, e.g. for identity and access management, specific system configuration like schedulers, filesystems, software management systems, and specific, sometimes vendor associated, toolchains, tooling, and container runtimes. In particular, the different scheduling policies encountered required us to adapt our workflows for each site. We found that having dedicated resources available, which was trialed in a period from 16th February to 14th April on LUMI, allowed to achieve high occupation rates, with 92% on the reserved GPU allocation and greater than 97% efficiency on the CPU reservation. Also, a stronger focus on federation of these systems, with a focus not only on federation of identities and accounts, but also in the areas of data ownership/transfer, observability, services and service accounts, maintenance coordination and performance portability could benefit DestinE. In general, it would be beneficial to make it easier to transfer a workload, or a digital twin system, from one EuroHPC site to the next and run and maintain them across several sites concurrently.

Energy hardware and workload aware job scheduling towards interconnected HPC environments

New HPC machines are getting close to the exascale. Power consumption for those machines has been increasing, and researchers are studying ways to reduce it. A second trend is HPC machines' growing complexity, with increasing heterogeneous hardware components and different clusters architectures cooperating in the same machine. We refer to these environments with the term heterogeneous multi-cluster environments. With the aim of optimizing performance and energy consumption in these environments, this paper proposes an Energy-Aware-Multi-Cluster (EAMC) job scheduling policy. EAMC-policy is able to optimize the scheduling and placement of jobs by predicting performance and energy consumption of arriving jobs for different hardware architectures and processor frequencies, reducing workload's energy consumption, makespan, and response time. The policy assigns a different priority to each job-resource combination so that the most efficient ones are favored, while less efficient ones are still considered on a variable degree, reducing response time and increasing cluster utilization. We implemented EAMC-policy in Slurm, and we evaluated a scenario in which two CPU clusters collaborate in the same machine. Simulations of workloads running applications modeled from real-world show a reduction of response time and makespan by up to 25% and 6% while saving up to 20% of total energy consumed when compared to policies minimizing runtime, and by 49%, 26%, and 6% compared to policies minimizing energy.

Dual mode scheduling in volunteer management

Nonprofit organizations have adopted online scheduling platforms that give autonomy to volunteers in the scheduling process. However, this strategy can create imbalances in task coverage, often requiring staff to fill the gaps. The aim of this study is to develop scheduling strategies to create a balanced schedule that effectively combines workforce types (paid staff and volunteers) while keeping volunteers engaged. This is achieved by accounting for volunteers’ responses to changes in scheduling options. We develop an optimization model that recognizes volunteers’ scheduling responses and utilizes these responses to design policies aimed at achieving a balanced coverage across time slots. This involves reducing over-covered and under-covered time slots over the planning horizon. By understanding the preferences of volunteers, organizations can modify their current policies to better match supply with demand keeping their volunteers engaged. We provide an implementable scheduling strategy combining staff assignment and volunteers’ autonomy in scheduling choices. Case study results show an improvement compared to current scheduling policies. Volunteers’ satisfaction increases, resulting in a long-term impact on the organizations and the communities they serve.

Deep Reinforcement Learning-Based Battery Management Algorithm for Retired Electric Vehicle Batteries with a Heterogeneous State of Health in BESSs

In this paper, we propose a battery management algorithm to optimize the lifetimes of retired lithium batteries with heterogeneous states of health in a battery energy storage system under dynamic power demand. A battery energy storage system allows for the use of retired lithium batteries for applications such as backup power in homes, data centers, etc. In a battery energy storage system, a battery pack consists of several retired batteries connected in parallel or in series to fulfill the required power demand. Owing to the retired batteries’ different capacity levels, i.e., states of health, a scheduling strategy is required to turn battery cells inside the battery pack on and off such that the secondary lifetimes of the retired batteries are extended. To establish the optimal scheduling policy, it is necessary to determine the correct states of each battery cell, including the state of charge and the state of health. To that end, the proposed algorithm first estimates the state of charge and state of health for all cells based on data measured using an extended Kalman filter. Then, a deep reinforcement learning scheduling algorithm is implemented to connect/disconnect the battery cells to/from the battery pack based on their states. Via simulation, we show that the proposed algorithm estimates the state of charge and state of health of each battery cell with low error and extends the lifetime of battery packs by 20.6%, compared to methods proposed in previous works.

Learning multi-satellite scheduling policy with heterogeneous graph neural network

In this paper, we investigate the multi-satellite scheduling (MSS) problem, a combinatorial optimization problem (COP) that aims to enhance satellite utilization. Numerous scholars have conducted in-depth studies on this matter. Handcrafted policies are commonly employed in various algorithms. Despite the use of graphical representations, designing policies manually becomes increasingly difficult, particularly as the number of satellites and tasks grows. Consequently, we propose a novel approach for automatically summarizing graph structures and learning policies for MSS problem. The proposed approach is developed based on graph neural network, Transformer, and sequence to sequence framework. First, MSS problem is modeled as a heterogeneous graph following various constraints. Next, the encoder, which is based on GNN and Transformer, is designed to summarize crucial information from the aforementioned graph. Finally, a two-stage decoder is proposed taking into account dynamic information in stepwise solution generation. Since obtaining optimal solutions in practical applications is challenging, we train the model using reinforcement learning. The results of the experiment demonstrate that our proposed method performs effectively in various scenarios. The quality of the solutions generated through this method transcends that of conventional construction methods. Additionally, the method generates solutions quickly, which are comparable in quality to those produced by meta-heuristic algorithms.

Characterizing Multi-Chip GPU Data Sharing

Multi-chip Graphics Processing Unit (GPU) systems are critical to scale performance beyond a single GPU chip for a wide variety of important emerging applications. A key challenge for multi-chip GPUs, though, is how to overcome the bandwidth gap between inter-chip and intra-chip communication. Accesses to shared data, i.e., data accessed by multiple chips, pose a major performance challenge as they incur remote memory accesses possibly congesting the inter-chip links and degrading overall system performance. This article characterizes the shared dataset in multi-chip GPUs in terms of (1) truly versus falsely shared data, (2) how the shared dataset scales with input size, (3) along which dimensions the shared dataset scales, and (4) how sensitive the shared dataset is with respect to the input’s characteristics, i.e., node degree and connectivity in graph workloads. We observe significant variety in scaling behavior across workloads: some workloads feature a shared dataset that scales linearly with input size, whereas others feature sublinear scaling (following a \(\sqrt {2}\) or \(\sqrt [3]{2}\) relationship). We further demonstrate how the shared dataset affects the optimum last-level cache organization (memory-side versus SM-side) in multi-chip GPUs, as well as optimum memory page allocation and thread scheduling policy. Sensitivity analyses demonstrate the insights across the broad design space.

Optimal energy allocation based on SINR under DoS attack

According to signal to interference plus noise ratio (SINR) of channel, we consider optimal attack schedule against remote state estimation in cyber–physical systems (CPSs) under denial of service (DoS) attack. Each physical process has a smart sensor that acquires the measurement of the system and transmits its local state estimation to the remote estimation terminal through a wireless channel. At each time step, a DoS attacker can send a jamming attack to one of the two channels. On the side of the attacker, this paper aims to search an optimal strategy, which maximizes the estimation error of the remote estimator. We propose an algorithm to obtain the optimal attack schedule policy by using of Markov decision process (MDP). A numerical example is provided to demonstrate the effectiveness of the proposed methods.

The application of heterogeneous graph neural network and deep reinforcement learning in hybrid flow shop scheduling problem

The hybrid flow shop scheduling problem (HFSP) is a fundamental optimization problem in the process industries. HFSP needs to consider both job selection and machine allocation. The existing HFSP solution methods represented by deep reinforcement learning (DRL) are challenging to extract complex state features and lack certain adaptability in dealing with large-scale HFSP. To solve HFSP with a large scale and relatively strong flexibility, we propose an end-to-end DRL architecture based on the heterogeneous graph neural network to learn an optimal scheduling policy. First of all, by adding a set of machine nodes, the disjunctive graph of the traditional flexible scheduling problem is extended and transformed into a heterogeneous graph, which intuitively represents the allocation relationship between operations and machines, and it is regarded as the scheduling state at different times; secondly, a new HFSP heterogeneous graph neural network (HGNN) algorithm is established to embed potential relationships between different types of nodes and edges, and an embedding of the entire graph is generated; finally, we use the graph-level representation vectors of heterogeneous graphs as input to the proximal policy optimization (PPO) algorithm for learning the optimal scheduling policy. The experimental results show that compared with several well-known heuristic algorithms, the scheduling policy obtained by the architecture through learning can have good generalization performance in simple and complex problems, respectively, and has a relatively high solution efficiency.

Integrated Stabilizing Control for Sampled-Data NCSs With Intermittent Observation and Multiple Random Transmission Delays

In this paper, we study sampled-data networked control systems (NCSs) that operate over communication networks with multi-path channels. Due to limited bandwidth power, the delivery of observed state signals from the sensor to the controller may suffer packet loss possibility, which leads to intermittent observation. An unreliable multi-path routing network is responsible for the transmission of control signals from the controller to the actuator, subject to random transmission delays. These communication problems lead to degradation in system performance, which may even cause instability. By utilizing an idle-time-based scheduling policy with a pre-defined deadline constraint on the actuator side, we transform the affiliated integrated system into a stochastic model with constant input delay. For such implementation, we propose a set of Riccati-type stabilization conditions, which are necessary and sufficient. It is remarkable that the stabilizing control policy is designed based on the feedback of the optimal estimate with minimum mean square error. Moreover, for the scalar system, we study the uniqueness and existence property of the maximum sampling period, and derive its close form representation in terms of system parameters, service capability and packet dropout rate.

Transaction Processing Policies in a Flexible Shuttle-based Storage and Re-trieval System by Real-time Data Tracking under Agent-based Modelling

This study investigates priority assignment rules (PARs) for transaction processing in automated warehouses featuring a shuttle-based storage and retrieval system (SBSRS). By incorporating real-time data tracking through agent-based modeling, the research explores the unique aspect of the SBSRS design, which involves flexible travel of robotic order picker shuttles be-tween tiers. The paper proposes PARs under agent-based modeling to enhance multi-objective performance metrics, including average flow time (AFT), maximum flow time (MFT), outlier transaction AFT, and standard deviations of flow times (SD) within the system. Experimental evaluations are conducted with various warehouse designs, comparing the results against commonly used static scheduling rules. The findings demonstrate that real-time tracking policies significantly improve system performance. Specifically, prioritizing the processing of outliers based on transaction waiting time enhances MFT, SD, and other performance metrics, while minimizing adverse effects on AFT. Certain rules exhibit notable improvements in MFT and SD, while others achieve the lowest AFT values among all experiments. This paper contributes to the existing literature by presenting a multi-objective performance improvement procedure and highlighting the advantages of real-time data track-ing-based scheduling policies in automated warehousing systems.

Age-Optimal Scheduling Over Hybrid Channels

We consider the problem of minimizing the age of information when a source can transmit status updates over two heterogeneous channels. Our work is motivated by recent developments in 5 G mmWave technology, where transmissions may occur over an unreliable but fast (e.g., mmWave) channel or a slow reliable (e.g., sub-6 GHz) channel. The unreliable channel is modeled as a time-correlated Gilbert-Elliot channel at a high rate when the channel is in the “ON” state. The reliable channel provides a deterministic but lower data rate. The scheduling strategy determines the channel to be used for transmission in each time slot, aiming to minimize the time-average age of information (AoI). The optimal scheduling problem is formulated as a Markov Decision Process (MDP), which is challenging to solve because super-modularity does not hold in a part of the state space. We address this challenge and show that a multi-dimensional threshold-type scheduling policy is optimal for minimizing the age. By exploiting the structure of the MDP and analyzing the discrete time Markov chains (DTMCs) of the threshold-type policy, we devise a low-complexity bisection algorithm to compute the optimal thresholds. We compare different scheduling policies using numerical simulations.

Distributed Energy Trading and Scheduling Among Microgrids via Multiagent Reinforcement Learning.

Renewable energy technologies empower microgrids to generate electricity to supply themselves and trade with others. Under this paradigm, microgrids have become autonomous entities that must intelligently determine their policies for energy trading and scheduling. Many factors influence a microgrid's decision-making, such as the complex microgrid infrastructure, the uncertain energy yield and demand, and the competition among the energy market players. These factors are usually hard to precisely model, and deriving the optimal policy for a microgrid is challenging. We propose a multiagent reinforcement learning (MARL) approach with an attention mechanism to learn the optimal policies for the microgrids without complex system modeling. We model each microgrid as an autonomous agent, which learns how to schedule energy resources and trade with others by collaborating with other agents. We adopt attention mechanism to enable intelligently selecting contextual information for the training of each agent. After training, an agent can make control decisions using only its local information, which can well preserve the microgrids' privacy and reduce the communication overhead among microgrids to facilitate distributed control. We implement a simulation environment and evaluate the performances of our proposed method using real-world datasets. The experimental results show that our method can significantly reduce the cost of the microgrids compared with the baseline methods.

Asynchronous Deep Reinforcement Learning for Collaborative Task Computing and On-Demand Resource Allocation in Vehicular Edge Computing

Vehicular Edge Computing (VEC) is enjoying a surge in research interest due to the remarkable potential to reduce response delay and alleviate bandwidth pressure. Facing the ever-growing service applications in VEC, how to effectively aggregate and flexibly schedule ubiquitous network resources for implementing diverse tasks and meeting differentiated demands from numerous vehicular users remains haunting. Toward this end, we investigate collaborative task computing and on-demand resource allocation. The collaborative computing framework in VEC is provided to support deep collaboration and intelligent management of heterogeneous resources widely distributed in vehicles, edge servers and cloud. Based on this framework, the joint optimization problem of distributed task offloading and multi-resource management is formulated with the aim to maximize the system utility by making the optimal task and resource scheduling policy, the novelty of which lies in the exploration of available vehicle resources and the consideration of service migration. In view of the dynamics, randomness and time-variant of vehicular networks, the asynchronous deep reinforcement algorithm is leveraged to find the optimal solution. Extensive simulation experiments are implemented to demonstrate the superiority of our proposed algorithm in terms of response latency compared with full offloading and random offloading.

SpecWands: An Efficient Priority-Based Scheduler Against Speculation Contention Attacks

Transient Execution Attacks (TEAs) have gradually become a major security threat to modern high-performance processors. They exploit the vulnerability of speculative execution to illegally access private data, and transmit them through timing-based covert channels. While new vulnerabilities are discovered continuously, the covert channels can be categorised to two types: 1) Persistent Type, in which covert channels are based on the layout changes of buffering, e.g. through caches or TLBs; 2) Volatile Type, in which covert channels are based on the contention of sharing resources, e.g. through execution units or issuing ports. The defenses against the persistent-type covert channels have been well addressed, while those for the volatile-type are still rather inadequate. Existing mitigation schemes for the volatile type such as Speculative Compression and Time-Division-Multiplexing will introduce significant overhead due to the need to stall the pipeline or to disallow resource sharing. In this paper, we look into such attacks and defenses with a new perspective, and propose a scheduling-based mitigation scheme, called SpecWands. It consists of three priority-based scheduling policies to prevent an attacker from transmitting the secret in different contention situations. SpecWands not only can defend against both inter-thread and intra-thread based attacks, but also can keep most of the performance benefit from speculative execution and resource-sharing. We evaluate its runtime overhead on SPEC 2017 benchmarks and realistic programs. The experimental results show that SpecWands has a significant performance advantage over the other two representative schemes.

Optimal Scheduling Policy for Time-Division Joint Radar and Communication Systems: Cross-Layer Design and Sensing for Free

Optimal Scheduling Policy for Time-Division Joint Radar and Communication Systems: Cross-Layer Design and Sensing for Free

Intelligent deep reinforcement learning-based scheduling in relay-based HetNets

We consider a fundamental file dissemination problem in a two-hop relay-based heterogeneous network consisting of a macro base station, a half-duplex relay station, and multiple users. To minimize the dissemination delay, rateless code is employed at the base station. Our goal is to find an efficient channel-aware scheduling policy at the half-duplex relay station, i.e., either fetch a packet from the base station or broadcast a packet to the users at each time slot, such that the file dissemination delay is minimized. We formulate the scheduling problem as a Markov decision process and propose an intelligent deep reinforcement learning-based scheduling algorithm. We also extend the proposed algorithm to adapt to dynamic network conditions. Simulation results demonstrate that the proposed algorithm performs very close to a lower bound on the dissemination delay and significantly outperforms baseline schemes.