Join Shortest Queue Research Articles

A Comprehensive Review of Load Balancing Techniques in Cloud Computing and Their Simulation with CloudSim Plus

Background: The field of cloud computing has been evolving for over a decade now. Load balancing is an important component of cloud computing. Load balancing implies scheduling of cloudlets (tasks) on virtual machines. Since this is an NP-hard problem, various heuristics for load balancing have been proposed in the research literature. The heuristics have been categorized, simulated and benchmarked in various ways; however, the information is scattered across many review articles. Objective: This review aims to bring a broad range of load balancing heuristics found in the research literature under one umbrella. It includes a comprehensive list of heuristics, a holistic set of criteria for their classification, and some key performance metrics and simulation tools used for their benchmarking. An illustration of a fair and comprehensive comparison of heuristics is provided using CloudSim Plus, a recent and advanced simulation tool. Method: The simulations performed with CloudSim Plus employ a generic model of task and machine heterogeneity with Poisson arrival of cloudlets and exponential distribution of cloudlet length to emulate actual cloud-computing scenarios. The simulation results in terms of key performance metrics are used to compare four centralized load balancing heuristics including Join Shortest Queue (JSQ), Join Idle Queue (JIQ), Round Robin and Minimum Completion Time (MCT).

TTLoC: Taming Tail Latency for Erasure-Coded Cloud Storage Systems

Distributed storage systems are known to be susceptible to long tails in response time. In modern online storage systems such as Bing, Facebook, and Amazon, the long tails of the service latency are of particular concern, with 99.9th percentile response times being orders of magnitude worse than the mean. As erasure codes emerge as a popular technique to achieve high data reliability in distributed storage while attaining space efficiency, taming tail latency still remains an open problem due to the lack of mathematical models for analyzing such systems. To this end, we propose a framework for quantifying and optimizing tail latency in erasure-coded storage systems. In particular, we derive upper bounds on tail latency in closed-form for arbitrary service time distribution and heterogeneous files. Based on the model, we formulate an optimization problem to jointly minimize weighted latency tail probability of all files over the placement of files on the servers, and the choice of servers to access the requested files. The non-convex problem is solved using an efficient, alternating optimization algorithm. Further, we mathematically quantify, in closed form, the tail index , i.e., the exponent at which latency tail probability diminishes to zero, of the service latency for arbitrary erasure-coded storage, by characterizing the asymptotic behavior of latency distribution tails. We further show that probabilistic scheduling-based algorithms are (asymptotically) optimal since they are able to achieve the exact tail index. Evaluation results show significant reduction of tail latency for erasure-coded storage systems with realistic workload. Based on the offline algorithm, an online version is developed and its superiority over the state-of-the-art algorithms, e.g., join-shortest-queue (JSQ), power-of-d [Pof(d))], least-load [LL(d)], is shown. Finally, a cloud storage system is implemented in a real cloud environment to show the superiority of our approach as compared to the considered baselines.

Designing Low-Complexity Heavy-Traffic Delay-Optimal Load Balancing Schemes

We establish a unified analytical framework for designing load balancing algorithms that can simultaneously achieve low latency, low complexity, and low communication overhead. We first propose a general class ¶ of load balancing policies and prove that they are both throughput optimal and heavy-traffic delay optimal. This class ¶ includes popular policies such as join-shortest-queue (JSQ) and power-of- d as special cases, but not the recently proposed join-idle-queue (JIQ) policy. In fact, we show that JIQ is not heavy-traffic delay optimal even for homogeneous servers. By exploiting the flexibility offered by the class ¶, we design a new load balancing policy called join-below-threshold (JBT-d), in which the arrival jobs are preferentially assigned to queues that are no greater than a threshold, and the threshold is updated infrequently. JBT-d has several benefits: (i) JBT-d belongs to the class ¶i and hence is throughput optimal and heavy-traffic delay optimal. (ii) JBT-d has zero dispatching delay, like JIQ and other pull-based policies, and low message overhead due to infrequent threshold updates. (iii) Extensive simulations show that JBT-d has good delay performance, comparable to the JSQ policy in various system settings.

Designing Low-Complexity Heavy-Traffic Delay-Optimal Load Balancing Schemes

In this paper, we establish a unified analytical framework for designing load balancing algorithms that can simultaneously achieve low latency, low complexity, and low communication overhead. We first propose a general class \Pi of load balancing policies and prove that they are throughput optimal and heavy-traffic delay optimal. This class \Pi includes popular policies such as join-shortest-queue (JSQ) and power-of- d as special cases, but not the recently proposed join-idle-queue (JIQ) policy. In fact, we show that JIQ is not heavy-traffic delay optimal even for homogeneous servers. By exploiting the flexibility offered by the class \Pi , we design a new load balancing policy called join-below-threshold (JBT-d), in which the arrival jobs are preferably assigned to queues that are no greater than a threshold, and the threshold is updated infrequently. JBT-d has several benefits: (i) JBT-d belongs to the class \Pi and hence is throughput optimal and heavy-traffic delay optimal. (ii) JBT-d has zero dispatching delay, like JIQ and other pull-based policies, and low message overhead due to infrequent threshold update. (iii) Extensive simulations show that JBT-d has excellent delay performance, comparable to the JSQ policy in various system settings.

On load balancing strategies for baggage screening at airports

We study load balancing policies for an airport baggage handling system (BHS). The performance of the screening subsystem is important as it can potentially cause traffic congestion of the whole BHS during peak period. Currently, the round-robin (RR) and the first-available (FA) polices are implemented at airports. This paper presents a simulative approach to evaluate the impact of load balancing policies on the system performance. Using discrete-event simulations, airport practitioners can assess the effectiveness of the policies according to the actual layout of a particular passenger terminal. In addition to the RR and FA, a join-shortest-queue (JSQ) policy is introduced. This scheme can be applied jointly with the RR for daily operations. A case study is presented to illustrate the characteristics of this scheme. Simulation results demonstrate that the RR-JSQ can replace the existing RR-FA for a better balanced load distribution and improve the overall system performance.

Load balancing for cluster systems under heavy-tailed and temporal dependent workloads

Large-scaled cluster systems have been employed in various areas by offering pools of fundamental resources. Efficient allocation of the shared resources in a cluster system is a critical but challenging issue, which has been extensively studied in the past few years. Despite the fact that existing load balancing policies, such as Random, Join Shortest Queue and size-based polices, are widely implemented in actual systems due to their simplicity and efficiency, the performance benefits of these policies diminish when workloads are highly variable and temporally correlated. In this paper, we propose a new load balancing policy, named ADuS, which attempts to partition jobs according to their present sizes and further rank the servers based on their loads. By dispatching jobs of similar sizes to the corresponding ranked servers, ADuS can adaptively balance user traffic and system load in a cluster and thus achieve significant performance benefits. Extensive trace-driven simulations using both synthetic and real traces show the effectiveness and robustness of ADuS under many different environments.

Cloud Computing Online Scheduling

Cloud computing has gained a lot of attention to be used as a computing model for a variety of application domains. Task scheduling is the fundamental issue in this environment. To utilize cloud efficiently, a good task scheduling algorithm is needed to assign tasks to resources in cloud. Cloud task can be divided into two categories such as on-line mode service and the batch mode service. In this paper, online cloud task scheduling based on virtual machine adaptive fault tolerance and load balancing using ant colony algorithm is proposed. The main contribution of this work is that load balancing factor is added and the system tolerates the faults by tacking the decision on the basis of reliability of the virtual machines in scheduling process. The proposed scheduling strategy was simulated using the Cloudsim toolkit package. Experimental results show that the proposed algorithm achieved the better load balance than Join-shortest-queue (JSQ) and Modified Ant Colony Optimization (MACO) algorithms.

Deferred Assignment Scheduling in Cluster-Based Servers

This paper proposes a new scheduling policy for cluster-based servers called DAS (Deferred Assignment Scheduling). The main idea in DAS is to defer scheduling as much as possible in order to make better use of the accumulated information on job sizes. In broad outline, DAS operates as follows: (1) incoming jobs are held by the dispatcher in a buffer; (2) the dispatcher monitors the number of jobs being processed by each server; (3) when the number of jobs at a server queue drops below a prescribed threshold, the dispatcher sends to it the shortest job in its buffer. To gauge the efficacy of DAS, the paper presents simulation studies, using various data traces. The studies collected response times and slowdowns for two cluster configurations under multi-threaded and multi-process back-end server architectures. The experimental results show that in both architectures, DAS outperforms the Round-Robin policy in all traffic regimes, and the JSQ (Join Shortest Queue) policy in medium and heavy traffic regimes.

JOIN MINIMUM COST QUEUE FOR MULTICLASS CUSTOMERS: STABILITY AND PERFORMANCE BOUNDS

We consider a system of K parallel queues providing different grades of service through each of the queues and serving a multiclass customer population. Service differentiation is achieved by specifying different join prices to the queues. Customers of class j define a cost function ψij(ci,xi) for taking service from queue i when the join price for queue i is ci and congestion in queue i is xi and join the queue that minimizes ψij(·,·). Such a queuing system will be called the “join minimum cost queue” (JMCQ) and is a generalization of the join shortest queue (JSQ) system. Non-work-conserving (called Paris Metro pricing system) and work-conserving (called the Tirupati system) versions of the JMCQ are analyzed when the cost to an arrival of joining a queue is a convex combination of the join price for that queue and the expected waiting time in that queue at the arrival epoch. Our main results are for a two-queue system.We obtain stability conditions and performance bounds. To obtain the lower and upper performance bounds, we propose two quasi-birth–death (QBD) processes that are derived from the original systems by suitably truncating the state space. The state space truncation in the non-work-conserving JMCQ follows the method of van Houtum and colleagues. We then show that this method is not applicable to the work-conserving JMCQ and provide sample-path-based proofs to show that the number in each queue is bounded by the number in the corresponding queues of these QBD processes. These sample-path proof techniques might also be of independent interest. We then show that the performance measures like mean queue length and revenue rate of the system are also bounded by the corresponding quantities of these QBD processes. Numerical examples show that these bounds are fairly tight. Finally, we generalize some of these results to systems with more queues.