MaxWeight Algorithm Research Articles

Heavy-traffic queue length behavior in a switch under Markovian arrivals

AbstractThis paper studies the input-queued switch operating under the MaxWeight algorithm when the arrivals are according to a Markovian process. We exactly characterize the heavy-traffic scaled mean sum queue length in the heavy-traffic limit, and show that it is within a factor of less than 2 from a universal lower bound. Moreover, we obtain lower and upper bounds that are applicable in all traffic regimes and become tight in the heavy-traffic regime.We obtain these results by generalizing the drift method recently developed for the case of independent and identically distributed arrivals to the case of Markovian arrivals. We illustrate this generalization by first obtaining the heavy-traffic mean queue length and its distribution in a single-server queue under Markovian arrivals and then applying it to the case of an input-queued switch.The key idea is to exploit the geometric mixing of finite-state Markov chains, and to work with a time horizon that is chosen so that the error due to mixing depends on the heavy-traffic parameter.

Tracking MaxWeight: Optimal Control for Partially Observable and Controllable Networks

Modern networks are complex and may include components that cannot be fully controlled or observed. Such network models can be characterized by overlay-underlay structures, where the network controller can only observe and operate on overlay nodes, and the underlay nodes are neither observable nor controllable. Classic network control algorithms may fail to work properly if they are only applied to the overlay nodes. To tackle this issue, we propose the Tracking MaxWeight (TMW*) algorithm that does not require direct observations of underlay nodes and only operates on overlay nodes. TMW* maintains virtual queues that track the dynamics of the underlay nodes and makes control decisions based on those virtual queues. We show that TMW* is throughput optimal as long as the network is stabilizable. We further extend our analysis to the setting that the estimates of the underlay state is erroneous and show that as long as the errors scale sub-linearly in time, TMW* preserves throughput optimality.

Back-Pressure traffic Signal Control in the Presence of Noisy Queue Information*

In this paper, we consider centralized traffic signal control policies using the max-weight algorithm when the queue size measurement is noisy. We first show analytically that the standard max-weight algorithm is throughput optimal even under noisy queue measurements. However, the average steady-state queue lengths and subsequently the average delays are increased. In order to alleviate the effect of these noisy measurements we add filtering to the max-weight algorithm; more specifically, we propose the Filtered-max-weight algorithm, which is based on particle filtering. We demonstrate via simulations that the Filtered-max-weight algorithm performs better than the standard max-weight algorithm in the presence of noisy measurements.

Logarithmic heavy traffic error bounds in generalized switch and load balancing systems

AbstractMotivated by applications to wireless networks, cloud computing, data centers, etc., stochastic processing networks have been studied in the literature under various asymptotic regimes. In the heavy traffic regime, the steady-state mean queue length is proved to be $\Theta({1}/{\epsilon})$ , where $\epsilon$ is the heavy traffic parameter (which goes to zero in the limit). The focus of this paper is on obtaining queue length bounds on pre-limit systems, thus establishing the rate of convergence to heavy traffic. For the generalized switch, operating under the MaxWeight algorithm, we show that the mean queue length is within $\textrm{O}({\log}({1}/{\epsilon}))$ of its heavy traffic limit. This result holds regardless of the complete resource pooling (CRP) condition being satisfied. Furthermore, when the CRP condition is satisfied, we show that the mean queue length under the MaxWeight algorithm is within $\textrm{O}({\log}({1}/{\epsilon}))$ of the optimal scheduling policy. Finally, we obtain similar results for the rate of convergence to heavy traffic of the total queue length in load balancing systems operating under the ‘join the shortest queue’ routeing algorithm.

Low-Complexity Switch Scheduling Algorithms: Delay Optimality in Heavy Traffic

Motivated by applications in data center networks, in this paper, we study the problem of scheduling in an input queued switch. While throughput maximizing algorithms in a switch are well-understood, delay analysis was developed only recently. It was recently shown that the well-known MaxWeight algorithm achieves optimal scaling of mean queue lengths in steady state in the heavy-traffic regime, and is within a factor less than $2$ of a universal lower bound. However, MaxWeight is not used in practice because of its high time complexity. In this paper, we study several low complexity algorithms and show that their heavy-traffic performance is identical to that of MaxWeight. We first present a negative result that picking a random schedule does not have optimal heavy-traffic scaling of queue lengths even under uniform traffic. We then show that if one picks the best among two matchings or modifies a random matching even a little, using the so-called flip operation, it leads to MaxWeight like heavy-traffic performance under uniform traffic. We then focus on the case of non-uniform traffic and show that a large class of low time complexity algorithms have the same heavy-traffic performance as MaxWeight, as long as it is ensured that a MaxWeight matching is picked often enough. We also briefly discuss the performance of these algorithms in the large scale heavy-traffic regime when the size of the switch increases simultaneously with the load. Finally, we perform empirical study on a new algorithm to compare its performance with some existing algorithms.

Learning to Schedule Network Resources Throughput and Delay Optimally Using Q+-Learning

As network architecture becomes complex and the user requirement gets diverse, the role of efficient network resource management becomes more important. However, existing throughput-optimal scheduling algorithms such as the max-weight algorithm suffer from poor delay performance. In this paper, we present reinforcement learning-based network scheduling algorithms for a single-hop downlink scenario which achieve throughput-optimality and converge to minimal delay. To this end, we first formulate the network optimization problem as a Markov decision process (MDP) problem. Then, we introduce a new state-action value function called Q <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> -function and develop a reinforcement learning algorithm called Q <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> -learning with UCB (Upper Confidence Bound) exploration which guarantees small performance loss during a learning process. We also derive an upper bound of the sample complexity in our algorithm, which is more efficient than the best known bound from Q-learning with UCB exploration by a factor of γ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> where γ is the discount factor of the MDP problem. Finally, via simulation, we verify that our algorithm shows a delay reduction of up to 40.8% compared to the max-weight algorithm over various scenarios. We also show that the Q <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> -learning with UCB exploration converges to an ε-optimal policy 10 times faster than Q-learning with UCB.

Minimizing the Energy Consumption of Wireless Sensor Network by Comparing the Performances of Maxweight and Minimum Energy Scheduling Algorithms

In this project we use MES algorithm is more favorable for dynamic link scheduling and network protocol designs in energy-constrained wireless networks such as wireless sensor networks which is well known to be throughput optimal. Before that we implement a novel scheduling method named Virtual Backbone Scheduling which schedules multiple overlapped backbones so that the network energy consumption is evenly distributed among all sensor nodes. In this way, the energy of all of the sensor nodes in the network is fully utilized, which in turn prolongs the network lifetime. Here the data is only forwarded by the backbone sensor nodes and other sensor nodes turn off the radios saves energy. The Energy Consumption is balanced by rotating the multiple backbones. Approximation algorithms are used for solving the MLBS problems. MES significantly reduces the energy consumption compared to the original MaxWeight algorithm. In addition, we analytically show that the MES algorithm is essentially energy optimal in the sense that the average energy expenditure of the MES algorithm can be pushed arbitrarily close to the global minimum solution.

Augmenting Max-Weight With Explicit Learning for Wireless Scheduling With Switching Costs

In small-cell wireless networks where users are connected to multiple base stations (BSs), it is often advantageous to switch off dynamically a subset of BSs to minimize energy costs. We consider two types of energy cost: (i) the cost of maintaining a BS in the active state, and (ii) the cost of switching a BS from the active state to inactive state. The problem is to operate the network at the lowest possible energy cost (sum of activation and switching costs) subject to queue stability. In this setting, the traditional approach -- a Max-Weight algorithm along with a Lyapunov-based stability argument -- does not suffice to show queue stability, essentially due to the temporal co-evolution between channel scheduling and the BS activation decisions induced by the switching cost. Instead, we develop a learning and BS activation algorithm with slow temporal dynamics, and a Max-Weight based channel scheduler that has fast temporal dynamics. We show using convergence of time-inhomogeneous Markov chains, that the co-evolving dynamics of learning, BS activation and queue lengths lead to near optimal average energy costs along with queue stability.

GB-PANDAS:

Dynamic affinity scheduling has been an open problem for nearly three decades. The problem is to dynamically schedule multi-type tasks to multi-skilled servers such that the resulting queueing system is both stable in the capacity region (throughput optimality) and the mean delay of tasks is minimized at high loads near the boundary of the capacity region (heavy-traffic optimality). As for applications, dataintensive analytics like MapReduce, Hadoop, and Dryad fit into this setting, where the set of servers is heterogeneous for different task types, so the pair of task type and server determines the processing rate of the task. The load balancing algorithm used in such frameworks is an example of affinity scheduling which is desired to be both robust and delay optimal at high loads when hot-spots occur. Fluid model planning, the MaxWeight algorithm, and the generalized c?-rule are among the first algorithms proposed for affinity scheduling that have theoretical guarantees on being optimal in different senses, which will be discussed in the related work section. All these algorithms are not practical for use in data center applications because of their non-realistic assumptions. The join-the-shortest-queue-MaxWeight (JSQMaxWeight), JSQ-Priority, and weighted-workload algorithms are examples of load balancing policies for systems with two and three levels of data locality with a rack structure. In this work, we propose the Generalized-Balanced-Pandas algorithm (GB-PANDAS) for a system with multiple levels of data locality and prove its throughput optimality. We prove this result under an arbitrary distribution for service times, whereas most previous theoretical work assumes geometric distribution for service times. The extensive simulation results show that the GB-PANDAS algorithm alleviates the mean delay and has a better performance than the JSQMaxWeight algorithm by up to twofold at high loads. We believe that the GB-PANDAS algorithm is heavy-traffic optimal in a larger region than JSQ-MaxWeight, which is an interesting problem for future work.

Optimal heavy-traffic queue length scaling in an incompletely saturated switch

We consider an input-queued switch operating under the MaxWeight scheduling algorithm. This system is interesting to study because it is a model for Internet routers and data center networks. Recently, it was shown that the MaxWeight algorithm has optimal heavy-traffic queue length scaling when all ports are uniformly saturated. Here we consider the case when an arbitrary number of ports are saturated (which we call the incompletely saturated case), and each port is allowed to saturate at a different rate. We use a recently developed drift technique to show that the heavy-traffic queue length under the MaxWeight scheduling algorithm has optimal scaling with respect to the switch size even in these cases.

On Achieving Fair and Throughput-Optimal Scheduling for TCP Flows in Wireless Networks

Throughput-optimal scheduling has been heavily investigated given its ability to fully utilize network resources and maintain network stability. Most of the existing throughput-optimal algorithms, including the classic queue-length based MaxWeight algorithm and flow-delay-based MaxWeight algorithm, however, may bring a severe unfairness problem when scheduling transmission control protocol (TCP) controlled flows. As TCP is the dominant transport layer protocol in the Internet and it controls the majority of Internet traffic, we study how to design the scheduling algorithm that can ensure both throughput optimality and be compatible to TCP flows. In this paper, we analyze the reason behind the incompatibility between the existing scheduling algorithms and TCP, and then investigate the properties of the head-of-line access delay-based scheduling algorithm (HOLD) we proposed. We prove that the proposed HOLD can fairly schedule TCP flows in wireless networks with time-varying channel conditions and achieve throughput optimality with flow-level dynamics. Simulations using OMNeT++ 4 have been conducted to validate our analytical results, and compare the performance of different scheduling algorithms comprehensively.

Throughput-Delay Trade-off Scheduling in Multi-channel Downlink Wireless Networks

This paper designs a down-link resource scheduling algorithm for wireless cellular networks to achieve throughput-delay trade-off. The available bandwidth of the down-link is divided into some parallel sub-channels, and each sub-channel can be allocated to one potential user in every time-slot. This problem is modeled as a multi-user multi-server discrete-time queuing system with a time-varying connectivity. For this system, it is well-known that the classical MaxWeight algorithm has the optimal throughput, but its delay performance is very poor. To overcome this issue, we use the Lyapunov Optimization technique to design a throughput-utility maximizing algorithm that provides a good trade-off between the throughput and delay performance. Our approach is verified by both theoretical analysis and simulation evaluations.

Optimal Heavy-Traffic Queue Length Scaling in an Incompletely Saturated Switch

We consider an input queued switch operating under the MaxWeight scheduling algorithm. This system is interesting to study because it is a model for Internet routers and data center networks. Recently, it was shown that the MaxWeight algorithm has optimal heavy-traffic queue length scaling when all ports are uniformly saturated. Here we consider the case where a fraction of the ports are saturated and others are not (which we call the incompletely saturated case), and also the case where the rates at which the ports are saturated can be different. We use a recently developed drift technique to show that the heavy-traffic queue length under the MaxWeight scheduling algorithm has optimal scaling with respect to the switch size even in these cases.

Heavy Traffic Queue Length Behavior in a Switch Under the MaxWeight Algorithm

We consider a switch operating under the MaxWeight scheduling algorithm, under any traffic pattern such that all the ports are loaded. This system is interesting to study since the queue lengths exhibit a multi-dimensional state-space collapse in the heavy-traffic regime. We use a Lyapunov-type drift technique to characterize the heavy-traffic behavior of the expectation of the sum queue lengths in steady-state, under the assumption that all ports are saturated and all queues receive non-zero traffic. Under these conditions, we show that the heavy-traffic scaled queue length is given by [Formula: see text], where σ is the vector of the standard deviations of arrivals to each port in the heavy-traffic limit. In the special case of uniform Bernoulli arrivals, the corresponding formula is given by [Formula: see text]. The result shows that the heavy-traffic scaled queue length has optimal scaling with respect to n, thus settling one version of an open conjecture; in fact, it is shown that the heavy-traffic queue length is at most within a factor of two from the optimal. We then consider certain asymptotic regimes where the load of the system scales simultaneously with the number of ports. We show that the MaxWeight algorithm has optimal queue length scaling behavior provided that the arrival rate approaches capacity sufficiently fast.

Dynamic scheduling with reconfiguration delays

We consider scheduling in networks with interference constraints and reconfiguration delays, which may be incurred when one service schedule is dropped and a distinct service schedule is adopted. Reconfiguration delays occur in a variety of communication settings, such as satellite, optical, or delay-tolerant networks. In the absence of reconfiguration delays it is well known that the celebrated Max-Weight scheduling algorithm guarantees throughput optimality without requiring any knowledge of arrival rates. As we will show, however, the Max-Weight algorithm may fail to achieve throughput optimality in case of nonzero reconfiguration delays. Motivated by the latter issue, we propose a class of adaptive scheduling algorithms which persist with the current schedule until a certain stopping criterion is reached, before switching to the next schedule. While earlier proposed Variable Frame-Based Max-Weight (VFMW) policies belong to this class, we also present Switching-Curve-Based (SCB) policies that are more adaptive to bursts in arrivals. We develop novel Lyapunov drift techniques to prove that this class of algorithms under certain conditions achieves throughput optimality by dynamically adapting the durations of the interswitching intervals. Numerical results demonstrate that these algorithms significantly outperform the ordinary Max-Weight algorithm, and that SCB policies yield a better delay performance than VFMW policies.

Heavy traffic queue length behavior in a switch under the MaxWeight algorithm

We consider a switch operating under the MaxWeight scheduling algorithm, under any traffic pattern such that all the ports are loaded. This system is interesting to study since the queue lengths ex...

MaxWeight Versus BackPressure: Routing and Scheduling in Multichannel Relay Networks

We study routing and scheduling algorithms for relay-assisted, multichannel downlink wireless networks [e.g., orthogonal frequency-division multiplexing (OFDM)-based cellular systems with relays]. Over such networks, while it is well understood that the BackPressure algorithm is stabilizing (i.e., queue lengths do not become arbitrarily large), its performance (e.g., delay, buffer usage) can be poor. In this paper, we study an alternative--the MaxWeight algorithm--variants of which are known to have good performance in a single-hop setting. In a general relay setting, however, MaxWeight is not even stabilizing (and thus can have very poor performance). In this paper, we study an iterative MaxWeight algorithm for routing and scheduling in downlink multichannel relay networks. We show that, surprisingly, the iterative MaxWeight algorithm can stabilize the system in several large-scale instantiations of this setting (e.g., general arrivals with full-duplex relays, bounded arrivals with half-duplex relays). Furthermore, using both many-channel large-deviations analysis and simulations, we show that iterative MaxWeight outperforms the BackPressure algorithm from a queue-length/delay perspective.

Heavy-Traffic Behavior of the MaxWeight Algorithm in a Switch with Uniform Traffic

We consider a switch with uniform traffic operating under the MaxWeight scheduling algorithm. This traffic pattern is interesting to study in the heavy-traffic regime since the queue lengths exhibit a multi-dimensional state-space collapse. We use a Lyapunov-type drift technique to characterize the heavy-traffic behavior of the expectation of the sum queue lengths in steady-state. Specifically, in the case of Bernoulli arrivals, we show that the heavy-traffic scaled queue length is ( n -- 3/2 + 1/2 n ). Our result implies that the MaxWeight algorithm has optimal queue-length scaling behavior in the heavy-traffic regime with respect to the size of a switch with a uniform traffic pattern. This settles the heavy-traffic version of an open conjecture.

MaxWeight Scheduling

The model is a "generalized switch", serving multiple traffic flows in discrete time. The switch uses MaxWeight algorithm to make a service decision (scheduling choice) at each time step, which determines the probability distribution of the amount of service that will be provided. We are primarily motivated by the following question: in the heavy traffic regime, when the switch load approaches critical level, will the service processes provided to each flow remain "smooth" (i.e., without large gaps in service)? Addressing this question reduces to the analysis of the asymptotic behavior of the unscaled queue-differential process in heavy traffic. We prove that the stationary regime of this process converges to that of a positive recurrent Markov chain, whose structure we explicitly describe. This in turn implies asymptotic "smoothness" of the service processes.

The Stability of Longest-Queue-First Scheduling With Variable Packet Sizes

It is well known that the MaxWeight scheduling algorithm is throughput-optimal in wireless networks. However, its complexity is exponential in the number of links in an ad hoc wireless network. In this work, we consider a greedy variant of the MaxWeight algorithm, called Longest Queue First (LQF) algorithm. A synchronous version of LQF algorithm is known to be throughput-optimal under a topological condition called local pooling. Here we study an asynchronous version which is suitable for implementation in networks with variable packet sizes. We show that the asynchronous LQF algorithm is also throughput-optimal under the local pooling condition.