Max-weight Scheduling Research Articles

Heavy Traffic Joint Queue Length Distribution withoutResource Pooling

This paper studies the Heavy Traffic (HT) joint distribution of queue lengths in an Input-queued switch (IQ switch) operating under the MaxWeight scheduling policy. IQ switchserve as representative of SPNs that do not satisfy the socalled Complete Resource Pooling (CRP) condition, and consequently exhibit a multidimensional State Space Collapse (SSC). Except in special cases, only mean queue lengths of such non-CRP systems is known in the literature. In this paper, we develop the Transform method to study the joint distribution of queue lengths in non-CRP systems. The key challenge is in solving an implicit functional equation involving the Laplace transform of the HT limiting distribution. For the general n x n IQ switch that has n2 queues, under a conjecture on uniqueness of the solution of the functional equation, we obtain an exact joint distribution of the HT limiting queue-lengths in terms of a non-linear combination of 2n iid exponentials.

Randomized Scheduling of Real-Time Traffic in Wireless Networks Over Fading Channels

Despite the rich literature on scheduling algorithms for wireless networks, algorithms that can provide deadline guarantees on packet delivery for general traffic and interference models are very limited. In this paper, we study the problem of scheduling real-time traffic under a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">conflict-graph interference model</i> with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unreliable links</i> due to channel fading. Packets that are not successfully delivered within their deadlines are of no value. We consider traffic (packet arrival and deadline) and fading (link reliability) processes that evolve as an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unknown</i> finite-state Markov chain. The performance metric is efficiency ratio which is the fraction of packets of each link which are delivered within their deadlines compared to that under the optimal (unknown) policy. We first show a conversion result that shows classical non-real-time scheduling algorithms can be ported to the real-time setting and yield a constant efficiency ratio. In particular, Max-Weight Scheduling () yields an efficiency ratio of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1/2$</tex-math> </inline-formula> . We then propose randomized algorithms that achieve efficiency ratios strictly higher than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1/2$</tex-math> </inline-formula> , by carefully randomizing over the maximal schedules. Further, we propose low-complexity and myopic distributed randomized algorithms, and characterize their efficiency ratio. Simulation results are presented that verify that the randomized algorithms outperform classical ones such as and for scheduling real-time traffic over fading channels.

Reinforcement Learning-Based Joint User Scheduling and Link Configuration in Millimeter-Wave Networks

In this paper, we develop algorithms for joint user scheduling and three types of mmWave link configuration: relay selection, codebook optimization, and beam tracking in millimeter wave (mmWave) networks. Our goal is to design an online controller that dynamically schedules users and configures their links to minimize system delay. To solve this complex scheduling problem, we model it as a dynamic decision-making process and develop two reinforcement learning-based solutions. The first solution is based on deep reinforcement learning (DRL), which leverages the proximal policy optimization to train a neural network-based solution. Due to the potential high sample complexity of DRL, we also propose an empirical multi-armed bandit (MAB)-based solution, which decomposes the decision-making process into a sequential of sub-actions and exploits classic maxweight scheduling and Thompson sampling to decide those sub-actions. Our evaluation of the proposed solutions confirms their effectiveness in providing acceptable system delay. It also shows that the DRL-based solution has better delay performance while the MAB-based solution has a faster training process.

Heavy Traffic Queue Length Distribution without Resource Pooling in an Input-Queued Switch

This paper studies the heavy-traffic joint distribution of queue lengths of an input-queued switch operating under the MaxWeight scheduling policy. Input-queued switch acts as a representative of SPNs that do not satisfy the so-called complete resource pooling (CRP) condition, and consequently exhibit a multidimensional state space collapse. Except in special cases, only mean queue lengths of such non-CRP systems have been obtained in the literature. In this paper, we develop the transform method to study the steady state distribution of non-CRP systems. The key challenge is in solving an implicit functional equation involving the Laplace transform of the heavy-traffic limiting distribution. We then consider the general n - n input-queued switch that has n2 queues. Under a conjecture on uniqueness of the solution of the functional equation, we obtain an exact joint distribution of the heavy-traffic limiting queue-lengths in terms of a nonlinear transformation of 2n iid exponentials.

Jumping Fluid Models and Delay Stability of Max-Weight Dynamics Under Heavy-Tailed Traffic

We say that a random variable is light-tailed if moments of order [Formula: see text] are finite for some [Formula: see text]; otherwise, we say that it is heavy-tailed. We study queueing networks that operate under the max-weight scheduling policy for the case in which some queues receive heavy-tailed and some receive light-tailed traffic. Queues with light-tailed arrivals are often delay stable (that is, expected queue sizes are uniformly bounded over time) but can also become delay unstable because of resource sharing with other queues that receive heavy-tailed arrivals. Within this context and for any given “tail exponents” of the input traffic, we develop a necessary and sufficient condition under which a queue is robustly delay stable, in terms of jumping fluid models—an extension of traditional fluid models that allows for jumps along coordinates associated with heavy-tailed flows. Our result elucidates the precise mechanism that leads to delay instability through a coordination of multiple abnormally large arrivals at possibly different times and queues and settles an earlier open question on the sufficiency of a particular fluid-based criterion. Finally, we explore the power of Lyapunov functions in the study of delay stability.

Two-Stage Self-Adaptive Task Outsourcing Decision Making for Edge-Assisted Multi-UAV Networks

This paper proposes a two-stage novel algorithm for intelligent edge-assisted multiple unmanned aerial vehicles (UAVs) surveillance services. In the first stage, multiple UAVs determine their optimal positions to detect as many target faces as possible for efficient surveillance using multi-agent deep reinforcement learning (MADRL). Multi-UAVs must be coordinated and optimally positioned for effective surveillance depending on the target person's location. It is also significantly important to consider the battery performance of the UAVs. In the second stage, every single UAV performs face identification in monitored areas, where two sequential scheduling methods make decisions: (i) edge selection for remote computing via max-weight scheduling and (ii) transmit power allocation scheduling to deliver the images to scheduled edges for time-average energy consumption minimization subject to stability. The identification execution requires computing power, and its complexity and time depend on the number of faces in the captured image. Consequently, the task cannot be fully executed by an individual UAV in high image arrival regimes or images with a high density of faces. In those conditions, UAVs can leverage computing support from nearby edges capable of AI-based face identification tasks. Importantly, computing task distribution should be energy-efficient and delay-minimal due to constraints imposed by the UAV system's characteristics and applications. We remark that selected edges should complete their computing tasks with similar delay to minimize idle time among the edges, which is why we chose min-max scheduling. To summarize, our proposed novel two-stage algorithm accomplishes efficient multi-UAV positioning corresponding to user-defined weight (overlapped threshold) and minimizes UAVs' transmission power while preserving stability constraints.

Learning-NUM: Network Utility Maximization With Unknown Utility Functions and Queueing Delay

Network Utility Maximization (NUM) studies the problems of allocating traffic rates to network users in order to maximize the users’ total utility subject to network resource constraints. In this paper, we propose a new NUM framework, Learning-NUM, where the users’ utility functions are unknown apriori and the utility function values of the traffic rates can be observed only after the corresponding traffic is delivered to the destination, which means that the utility feedback experiences queueing delay. The goal is to design a policy that gradually learns the utility functions and makes rate allocation and network scheduling/routing decisions so as to maximize the total utility obtained over a finite time horizon <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> . In addition to unknown utility functions and stochastic constraints, a central challenge of our problem lies in the queueing delay of the observations, which may be unbounded and depends on the decisions of the policy. We first show that the expected total utility obtained by the best dynamic policy is upper bounded by the solution to a static optimization problem. Without the presence of feedback delay, we design an algorithm based on the ideas of gradient estimation and Max-Weight scheduling. To handle the feedback delay, we embed the algorithm in a parallel-instance paradigm to form a policy that achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\tilde {O}(T^{3/4})$ </tex-math></inline-formula> -regret, i.e., the difference between the expected utility obtained by the best dynamic policy and our policy is in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\tilde {O}(T^{3/4})$ </tex-math></inline-formula> . Furthermore, we extend our policy to deal with the case where the utility observations are noisy and show that it achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\tilde {O}(T^{7/8})$ </tex-math></inline-formula> -regret. Finally, to demonstrate the practical applicability of the Learning-NUM framework, we apply it to three application scenarios including database query, job scheduling and video streaming. We further conduct simulations on the job scheduling application to evaluate the empirical performance of our policy.

On Heavy-Traffic Optimal Scaling of c-Weighted MaxWeight Scheduling in Input-Queued Switches

We consider the asymptotically optimal control of input-queued switches under a cost-weighted variant of MaxWeight scheduling, for which we establish theoretical properties that include showing the algorithm exhibits optimal heavy-traffic queue-length scaling. Our results are expected to be of theoretical interest more broadly than input-queued switches.

Optimal Control in Fluid Models of nxn Input-Queued Switches under Linear Fluid-Flow Costs

We consider a fluid model of n x n input-queued switches with associated fluid-flow costs and derive an optimal scheduling control policy to an infinite horizon discounted control problem with a general linear objective function of fluid cost. Our optimal policy coincides with the cμ-rule in certain parameter domains, but more generally, takes the form of the solution to a flow maximization problem. Computational experiments demonstrate the benefits of our optimal scheduling policy over variants of max-weight scheduling and the cμ-rule.

Heavy-traffic Analysis of the Generalized Switch under Multidimensional State Space Collapse

Stochastic Processing Networks that model wired and wireless networks, and other queueing systems, have been studied in heavytraffic limit under the so-called Complete Resource Pooling (CRP) condition. When the CRP condition is not satisfied, heavy-traffic results are known only in the special case of an input-queued switch and bandwidth-sharing network. In this paper, we consider a very general queueing system called the 'generalized switch' that includes wireless networks under fading, data center networks, input-queued switch, etc. The primary contribution of this paper is to present the exact value of the steadystate mean of certain linear combinations of queue lengths in the heavy-traffic limit under MaxWeight scheduling algorithm. We use the Drift method, and we also present a negative result that it is not possible to obtain the remaining linear combinations (and consequently all the individual mean queue lengths) using this method. We do this by presenting an alternate view of the Drift method in terms of an (under-determined) system of linear equations. Finally, we use this system of equations to obtain upper and lower bounds on all linear combinations of queue lengths.

Fluctuation Bounds for the Max-Weight Policy with Applications to State Space Collapse

We consider a multihop switched network operating under a max-weight scheduling policy and show that the distance between the queue length process and a fluid solution remains bounded by a constant multiple of the deviation of the cumulative arrival process from its average. We then exploit this result to prove matching upper and lower bounds for the time scale over which additive state space collapse (SSC) takes place. This implies, as two special cases, an additive SSC result in diffusion scaling under non-Markovian arrivals and, for the case of independent and identically distributed arrivals, an additive SSC result over an exponential time scale.

Throughput Maximization in Uncooperative Spectrum Sharing Networks

Throughput-optimal transmission scheduling in wireless networks has been a well considered problem in the literature, and the method for achieving optimality, MaxWeight scheduling, has been known for several decades. This algorithm achieves optimality by adaptively scheduling transmissions relative to each user’s stochastic traffic demands. To implement the method, users must report their queue backlogs to the network controller and must rapidly respond to the resulting resource allocations. However, many currently-deployed wireless systems are not able to perform these tasks and instead expect to occupy a fixed assignment of resources. To accommodate these limitations, adaptive scheduling algorithms need to interactively estimate these uncooperative users’ queue backlogs and make scheduling decisions to account for their predicted behavior. In this work, we address the problem of scheduling with uncooperative legacy systems by developing algorithms to accomplish these tasks. We begin by formulating the problem of inferring the uncooperative systems’ queue backlogs as a partially observable Markov decision process and proceed to show how our resulting learning algorithms can be successfully used in a queue-length-based scheduling policy. Our theoretical analysis characterizes the throughput-stability region of the network and is verified using simulation results.

Heavy-traffic Analysis of the Generalized Switch under Multidimensional State Space Collapse

Stochastic Processing Networks that model wired and wireless networks, and other queueing systems, have been studied in heavytraffic limit under the so-called Complete Resource Pooling (CRP) condition. When the CRP condition is not satisfied, heavy-traffic results are known only in the special case of an input-queued switch and bandwidth-sharing network. In this paper, we consider a very general queueing system called the 'generalized switch' that includes wireless networks under fading, data center networks, input-queued switch, etc. The primary contribution of this paper is to present the exact value of the steadystate mean of certain linear combinations of queue lengths in the heavy-traffic limit under MaxWeight scheduling algorithm. We use the Drift method, and we also present a negative result that it is not possible to obtain the remaining linear combinations (and consequently all the individual mean queue lengths) using this method. We do this by presenting an alternate view of the Drift method in terms of an (under-determined) system of linear equations. Finally, we use this system of equations to obtain upper and lower bounds on all linear combinations of queue lengths.

On Scheduling Policies With Heavy-Tailed Dynamics in Wireless Queueing Systems

This paper takes a system view and studies a wireless queueing system where heavy-tailness may occur both at the traffic arrival and in the form of the multi-user interference. With the rapid development of AI technologies, this heavy-tailed traffic model has become more prevalent in the current network system, such as the file or data size used in the deep learning algorithm. We first re-visit the standard asymmetric queueing system with a mix of heavy-tailed and light-tailed traffic, but under a new variable-rate service model that not only better models the dynamics of the wireless medium but also includes the previous models as special cases. We then focus on the scheduling problem when heavy-tailed interference disrupts the serving link. The performance of queueing policies is investigated during an ON/OFF renewal channel process with heavy-tailed OFF periods, and the expected queue length and the throughput characteristic is studied under the priority as well as max-weight scheduling policies. The results show that the expected queue length of the heavy queue cannot be maintained as finite even under the most favorable priority policy. On the other hand, a priority policy can guarantee the finiteness of an expected queue length for the light queue, but the system is not throughput optimal any longer. It is further shown that no benefit can be provided by the max-weight scheduling policy to the light queue for the queue length behavior in a steady-state, though the system is always throughput optimal.

Learn-as-you-go with Megh: Efficient Live Migration of Virtual Machines

Cloud providers leverage live migration of virtual machines to reduce energy consumption and allocate resources efficiently in data centers. Each migration decision depends on three questions: when to move a virtual machine, which virtual machine to move and where to move it? Dynamic, uncertain, and heterogeneous workloads running on virtual machines make such decisions difficult. Knowledge-based and heuristics-based algorithms are commonly used to tackle this problem. Knowledge-based algorithms, such as MaxWeight scheduling algorithms, are dependent on the specifics and the dynamics of the targeted Cloud architectures and applications. Heuristics-based algorithms, such as MMT algorithms, suffer from high variance and poor convergence because of their greedy approach. We propose an online reinforcement learning algorithm called Megh. Megh does not require prior knowledge of the workload rather learns the dynamics of workloads as-it-goes. Megh models the problem of energy- and performance-efficient resource management during live migration as a Markov decision process and solves it using a functional approximation scheme. While several reinforcement learning algorithms are proposed to solve this problem, these algorithms remain confined to the academic realm as they face the curse of dimensionality. They are either not scalable in real-time, as it is the case of MadVM, or need an elaborate offline training, as it is the case of Q-learning. These algorithms often incur execution overheads which are comparable with the migration time of a VM. Megh overcomes these deficiencies. Megh uses a novel dimensionality reduction scheme to project the combinatorially explosive state-action space to a polynomial dimensional space with a sparse basis. Megh has the capacity to learn uncertain dynamics and the ability to work in real-time without incurring significant execution overhead. Megh is both scalable and robust. We implement Megh using the CloudSim toolkit and empirically evaluate its performance with the PlanetLab and the Google Cluster workloads. Experiments validate that Megh is more cost-effective, converges faster, incurs smaller execution overhead and is more scalable than MadVM and MMT. An empirical sensitivity analysis explicates the choice of parameters in experiments.

Sensitivity to Cumulative Perturbations for a Class of Piecewise Constant Hybrid Systems

We consider a class of continuous-time hybrid dynamical systems that correspond to subgradient flows of a piecewise linear and convex potential function with finitely many pieces, and which includes the fluid-level dynamics of the Max-Weight scheduling policy as a special case. We study the effect of an external disturbance/perturbation on the state trajectory, and establish that the magnitude of this effect can be bounded by a constant multiple of the integral of the perturbation.

A Novel Scheduling Approach for Packet Delay in Wireless Ad Hoc Networks and Comparative Analysis with Other Scheduling Approaches

Objectives: The goal of the work is to design a network as well as a new scheduling approach that provides minimizing the delay, better operation of resources in the network and increasing optimality. Methods: In the wireless network and wireline Network, the delay stability is one of the significant metrics considered. The delay analysis of packets plays an essential job in the network. Decomposition of packets partition into multiple paths, checks whether any two nodes assemble at same point as bottleneck occurred or not. To overcome these issues, it utilizes a new delay efficient scheduling policy. Findings: In this research, a new scheduling algorithm, Ant Colony Optimization Max Weight Scheduling (ACOMWS) for minimizing the delay, is proposed. For recognizing the behavior of each path in the network, lower bound analysis is used. Comparative result analyses of proposed scheduling algorithm (ACOMWS) are evaluated with other algorithms (MWS, BP, ACO, Greedy, and GMWS). Application/Improvements: The proposed hybrid scheduling algorithm (ACOMWS) improves the results to achieve better precision results compared to the previous related works. Keywords: Adhoc, Backpressure, Greedy, Max-weight, Packet Delay, Scheduling, Throughput Optimal

Wireless Scheduling for Information Freshness and Synchrony: Drift-Based Design and Heavy-Traffic Analysis

We consider the problem of scheduling in wireless networks with the aim of maintaining up-to-date and synchronized (also called, aligned ) information at the receiver across multiple flows. This is in contrast to the more conventional approach of scheduling for optimizing long-term performance metrics such as throughput, fairness, or average delay. Maintaining the age of information at a low and roughly equal level is particularly important for distributed cyber-physical systems, in which the effectiveness of the control decisions depends critically on the freshness and synchrony of information from multiple sources/sensors. In this paper, we first expose the weakness of several popular MaxWeight scheduling solutions that utilize queue-length, delay, and age information as their weights. Then, we develop a novel age-based scheduler that combines age with the interarrival times of incoming packets in its decisions, which yields significant gains in the information freshness at the receiver. We characterize the performance of our strategy through a heavy-traffic analysis that establishes upper and lower bounds on the freshness of system information.

Traffic-Aware Scheduling and Feedback Allocation in Multichannel Wireless Networks

This paper studies the problem of feedback allocation and scheduling for a multichannel downlink cellular network under limited and delayed feedback. We propose two efficient algorithms that select the link states that should be reported to the base station (BS). A novelty here is that these feedback allocation algorithms are performed at the users’ side to take advantage of their local channel state information knowledge in order to achieve higher gains. The first algorithm is suitable for a continuous-time contention scheme and requires only one feedback per channel, whereas the second one is adapted for a discrete-time contention scheme and adopts a threshold-based concept. For this second algorithm, we study some implementation aspects related to the feedback period and investigate the tradeoff between knowing at the BS a small number of accurate link states and a larger but outdated number of link states. We show that these algorithms, combined with the Max-Weight scheduling, achieve good stability performance compared with the ideal system.

An Optimal Scheduling Policy for the 2 X 2 Input-Queued Switch with Symmetric Arrival Rates

We investigate a cannonical input-queued switch scheduling problem in which the objective is to minimize the infinite horizon discounted queue length under symmetric arrivals, for which we derive an optimal scheduling policy and establish its theoretical properties with respect to delay. We then compare via simulation these theoretical properties of our optimal policy with those of the well-known MaxWeight scheduling algorithm in order to gain insights on the delay optimality of the MaxWeight scheduling policy.