Homogeneous Servers Research Articles

Designing Low-Complexity Heavy-Traffic Delay-Optimal Load Balancing Schemes

We establish a unified analytical framework for designing load balancing algorithms that can simultaneously achieve low latency, low complexity, and low communication overhead. We first propose a general class ¶ of load balancing policies and prove that they are both throughput optimal and heavy-traffic delay optimal. This class ¶ includes popular policies such as join-shortest-queue (JSQ) and power-of- d as special cases, but not the recently proposed join-idle-queue (JIQ) policy. In fact, we show that JIQ is not heavy-traffic delay optimal even for homogeneous servers. By exploiting the flexibility offered by the class ¶, we design a new load balancing policy called join-below-threshold (JBT-d), in which the arrival jobs are preferentially assigned to queues that are no greater than a threshold, and the threshold is updated infrequently. JBT-d has several benefits: (i) JBT-d belongs to the class ¶i and hence is throughput optimal and heavy-traffic delay optimal. (ii) JBT-d has zero dispatching delay, like JIQ and other pull-based policies, and low message overhead due to infrequent threshold updates. (iii) Extensive simulations show that JBT-d has good delay performance, comparable to the JSQ policy in various system settings.

Two server Markovian inventory systems with server interruptions: Heterogeneous vs. homogeneous servers

This paper discusses about Markovian (s,Q) inventory-queueing system with server interruptions of two heterogeneous servers. Matrix method is applied to obtain the steady state joint probability distribution of customers level in the queue, retrial group, status of the servers and stock level of heterogeneous system. This paper studies the significant effect of heterogeneous system (each server work with different service rate) as compared with homogeneous system (all servers work with same service rate). Performance measures are derived for heterogeneous system and homogeneous system. Appropriate cost functions for both models are defined. Numerical examples are used to compare the total expected cost rates of both systems. Heterogeneous system yields the minimum total expected cost value and therefore the heterogeneous system is effective.

Towards the Analysis of the Queuing System Operating in the Random Environment with Resource Allocation

The mathematical model of the system, that consists of a storage device and several homogeneous servers and operates in a random environment, and provides incoming applications not only services, but also access to resources of the system, is being constructed. The random environment is represented by two independent Markov processes. The first of Markov processes controls the incoming flow of applications to the system and the size of resources required by each application. The incoming flow is a Poisson one, the rate of the flow and the amount of resources required for the application are determined by the state of the external Markov process. The service time for applications on servers is exponential distributed. The service rate and the maximum amount of system resources are determined by the state of the second external Markov process. When the application leaves the system, its resources are returned to the system. In the system under consideration, there may be failures in accepting incoming applications due to a lack of resources, as well as loss of the applications already accepted in the system, when the state of the external Markov process controlling the service and provision of resources changes. A random process describing the functioning of this system is constructed. The system of equations for the stationary probability distribution of the constructed random process is presented in scalar form. The main tasks for further research are formulated.

Performance and cost analysis of a finite capacity queue with two heterogeneous servers under F-policy

All previous works on F-policy queues with multiple servers have focused on multiple homogeneous servers. This paper presents an analytical analysis on the F-policy M/M/2/K queue with heterogeneous servers. We obtained the steady-state distribution of the number of customers in the system and developed a variety of performance measures. The expected cost function was constructed to determine the optimal threshold value and optimal service rates with the objective of minimising the expected cost per unit time. To this end, we developed an algorithm based on the quasi-Newton method to solve the problem of cost minimisation. Finally, sensitivity analysis was conducted using numerical examples.

Designing Low-Complexity Heavy-Traffic Delay-Optimal Load Balancing Schemes

In this paper, we establish a unified analytical framework for designing load balancing algorithms that can simultaneously achieve low latency, low complexity, and low communication overhead. We first propose a general class \Pi of load balancing policies and prove that they are throughput optimal and heavy-traffic delay optimal. This class \Pi includes popular policies such as join-shortest-queue (JSQ) and power-of- d as special cases, but not the recently proposed join-idle-queue (JIQ) policy. In fact, we show that JIQ is not heavy-traffic delay optimal even for homogeneous servers. By exploiting the flexibility offered by the class \Pi , we design a new load balancing policy called join-below-threshold (JBT-d), in which the arrival jobs are preferably assigned to queues that are no greater than a threshold, and the threshold is updated infrequently. JBT-d has several benefits: (i) JBT-d belongs to the class \Pi and hence is throughput optimal and heavy-traffic delay optimal. (ii) JBT-d has zero dispatching delay, like JIQ and other pull-based policies, and low message overhead due to infrequent threshold update. (iii) Extensive simulations show that JBT-d has excellent delay performance, comparable to the JSQ policy in various system settings.

Performance Degradation in Parallel-Server Systems

We consider a parallel-server system with homogeneous servers where incoming tasks, arriving at rate $\lambda $ , are dispatched by $n$ dispatchers, each of them balancing a fraction $1/n$ of the load to $K/n$ servers. Servers are first-come-first-served (FCFS) queues and dispatchers implement size interval task assignment policy with equal load (SITA-E), a size-based policy such that the servers are equally loaded. We compare the performance of a system with $n>1$ dispatchers and a single dispatcher. We show that the performance of a system with $n$ dispatchers, $K$ servers, and arrival rate $\lambda $ coincides with that of a system with one dispatcher, $K/n$ servers, and arrival rate $\lambda /n$ . We define the degradation factor as the ratio between the performance of a system with $K$ servers and arrival rate $\lambda $ and the performance of a system with $K/n$ servers and arrival rate $\lambda /n$ . We establish a partial monotonicity on $n$ for the degradation factor and, therefore, the degradation factor is lower bounded by one. We then investigate the upper bound of the degradation factor for particular distributions. We consider two continuous service time distributions: uniform and bounded Pareto and a discrete distribution with two values, which is the distribution that maximizes the variance for a given mean. We show that the performance degradation is small for uniformly distributed job sizes but that for Bounded Pareto and two points distributions it can be unbounded. We have investigated the degradation using the distribution obtained from real traces.

Boosting Parallel File System Performance via Heterogeneity-Aware Selective Data Layout

Hybrid parallel file systems (PFS) that combine HDD servers with SSD servers provide a promising solution for data intensive applications. The efficiency of a hybrid PFS relies on the data layout schemes. However, most current layout strategies are designed for homogeneous servers, which neither address the heterogeneity of servers nor the varying access patterns of applications. In this paper, we propose HAS, a novel heterogeneity-aware selective data layout scheme for hybrid PFSs. HAS alleviates inter-server load imbalance through skewing data distribution on heterogeneous servers based on their storage performance. Furthermore, to obtain the optimal performance for a specific access pattern, HAS selects one static data layout policy with lowest access cost from three typical layout candidates as the final file data layout method. To adapt to the mixed access patterns within an application, HAS uses a dynamic data layout scheme, which stores file with multiple copies, each using a different data layout policy, and then selects the copy with the lowest access cost to serve file requests. We have implemented HAS within MPICH2 and OrangeFS. Experimental results show that HAS can significantly increase the I/O throughput of hybrid PFSs, compared to existing data layout optimization methods.

Allocation of flows in closed bipartite queueing networks

This paper describes a novel method for allocating agents to routes in a closed bipartite queueing network to maximize system throughput using three open network approximations. Results are presented which compare this method with known prior work and optimal solutions to provide an empirical optimality gap. Average empirical optimality gaps of 1.29 percent, 1.13 percent and 1.29 percent are observed for the three approximations considered. Further, because many systems are under the control of rational agents, conditions are derived in order to determine properties of the market context that induce optimal behavior. It is shown that uniform rewards do not yield an efficient rational equilibrium in general. However, for systems with homogeneous servers and travel times or those with travel times that are much larger than queue waiting times, uniform rewarding is optimal.

Energy management for the homogeneous server clusters offering web services

The population of the Internet users has exceeded 2.4 billion. Data centers provide Internet services to fulfill the demand from these users. Many data centers adopt the cluster-based server systems to host the required Internet services. These server systems consume significant amount of energy, but much of the power is used to maintain service capacity during idle or low workload periods. This paper surveys some recent approaches addressing this issue. As learned from the traditional telephone call center planning processes, a queueing model is adopted to model the server clusters with homogeneous architecture. By analyzing the model, a set of the factors affecting the energy cost is identified. Based on the identified factors, an on-line energy management technique is then deigned. The proposed approach is simulated with a real-world workload trace. The simulation result shows that approximately 70 to 75 % of the originally consumed energy can be saved.

Tollbooth tandem queues with infinite homogeneous servers

In this paper we analyze a tollbooth tandem queueing problem with an infinite number of servers. A customer starts service immediately upon arrival but cannot leave the system before all customers who arrived before him/her have left, i.e. customers depart the system in the same order as they arrive. Distributions of the total number of customers in the system, the number of departure-delayed customers in the system, and the number of customers in service at time t are obtained in closed form. Distributions of the sojourn times and departure delays of customers are also obtained explicitly. Both transient and steady state solutions are derived first for Poisson arrivals, and then extended to cases with batch Poisson and nonstationary Poisson arrival processes. Finally, we report several stochastic ordering results on how system performance measures are affected by arrival and service processes.

Tollbooth tandem queues with infinite homogeneous servers

In this paper we analyze a tollbooth tandem queueing problem with an infinite number of servers. A customer starts service immediately upon arrival but cannot leave the system before all customers who arrived before him/her have left, i.e. customers depart the system in the same order as they arrive. Distributions of the total number of customers in the system, the number of departure-delayed customers in the system, and the number of customers in service at time t are obtained in closed form. Distributions of the sojourn times and departure delays of customers are also obtained explicitly. Both transient and steady state solutions are derived first for Poisson arrivals, and then extended to cases with batch Poisson and nonstationary Poisson arrival processes. Finally, we report several stochastic ordering results on how system performance measures are affected by arrival and service processes.

FAMA: A Middleware for Fast Deploying and Auto Scaling towards Multitier Applications in Clouds

As the scale of Internet applications increases, the structure and technology used in backend application servers keep on improving. Facing multitier applications with large workload, how to deploy them rapidly and maintain them effectively become challenges. The traditional solution widely adopted is to make up a large physical cluster of homogeneous servers to combine individual computational capability to improve application performance. However, this way has little flexibility for dynamical load, and finally leads to low recourse utilization and delayed reaction. Relatively, though cloud computing provides a more flexibility and scalability model, traditional cloud is still not convenient and intelligent enough to deploy and maintain applications. We present a fast-deploying and auto-scaling middleware named FAMA towards multitier applications in cloud. FAMA solves the fast-deploying problem on one hand, and provides an open and flexible mechanism to scale applications automatically to meet load peak on the other hand. We implement the prototype and combine it within OpenNebula for evaluation. Our work shows that FAMA can deploy and scale applications in a very short time. Comparing to original IaaS platforms, it shortens the VM-creation to only few seconds. Through compatible with users' own monitoring mechanism, FAMA has improved flexibility and scalability.

Metaheuristics for reliable server assignment problems

Previous studies of reliable server assignment considered only to assign the same cost of server, that is, homogeneous servers. In this paper, we generally deal with reliable server assignment with different server costs, i.e., heterogeneous servers. We formulate this problem as a nonlinear integer programming mathematically. Our problem is defined as determining a deployment of heterogeneous servers to maximize a measure of service availability. We propose two metaheuristic algorithms (tabu search and particle swarm optimization) for solving the problem of reliable server assignment. From the computational results, we notice that our tabu search outstandingly outperforms particle swarm optimization for all test problems. In terms of solution quality and computing time, the proposed method is recommended as a promising metaheuristic for a kind of reliability optimization problems including reliable sever assignment and e-Navigation system.

Multiserver systems with rejections and the equiprobable distribution of customers

The queueing systems with rejections, arbitrary service time distributions, and two ways of the distributions of customers over heterogeneous servers (the equiprobable customer distribution over all (idle and busy) servers and the equiprobable customer distribution over idle servers) are discussed and used to find the stationary distributions of the number of customers. Their invariance with respect to service time distribution functions is proved. For an MX/M/n/0 system with homogeneous servers and the equiprobable customer distribution over all servers, the algorithm is proposed for determining the stationary distributions of the number of customers when their number in the batch is less than six. MX/M/2/0 and MX/M/3/0 systems with the equiprobable customer distribution over all servers and the arbitrary distribution of the number of arriving customers are studied. The results are checked using simulation models based on GPSS World software.

Runtimes and optimizations for MapReduce

Map Reduce is emerging as an important programming model for data-intensive applications. It is widely used in a variety of different environments to deal with the services of big data processing. In order to adapt to these platforms, the original implementation of Map Reduce framework has been extended and enhanced. This paper describes a comprehensive taxonomy of Map Reduce runtimes for different computing environments ranging from traditional homogeneous PC cluster server to dynamic, virtual, heterogeneous, and mobile computing platform, aiming at a good understanding of wide applicability of this model. It also provides a survey for the optimization strategies applied in these runtimes and that are gaining a lot of momentum in both research and industrial communities. The taxonomy and survey is used to identify and expand the fields of application for MapReduce model and provide references for research.

Modelling activities at a neurological rehabilitation unit

A queuing model of a specialist neurological rehabilitation unit is studied. The application is to the Neurological Rehabilitation Centre at Rookwood Hospital (Cardiff, UK), the national rehabilitation unit for Wales. Due to high demand this 21-bed inpatient facility is nearly always at maximum occupancy, and with a significant bed-cost per day this makes it a prime candidate for mathematical modelling. Central to this study is the concept that treatment intensity has an effect on patient length of stay. The model is constructed in four stages. First, appropriate patient groups are determined based on a number of patient-related attributes. Second, a purpose-built scheduling program is used to deduce typical levels of treatment to patients of each group. These are then used to estimate the mean length of stay for each patient group. Finally, the queuing model is constructed. This consists of a number of disconnected homogeneous server queuing systems; one for each patient group. A Coxian phase-type distribution is fitted to the length of time from admission until discharge readiness and an exponential distribution models the remainder of time until discharge. Some hypothetical scenarios suggested by senior management are then considered and compared on the grounds of a number of performance measures and cost implications.

Two Homogeneous Servers Limited Capacity Markovian Queuing System Subjected to Varying Catastrophic Intensity

Two Homogeneous Servers Limited Capacity Markovian Queuing System Subjected to Varying Catastrophic Intensity

A Recurrent Solution of PH/M/c/N-Like and PH/M/c-Like Queues

We propose an efficient semi-numerical approach to compute the steady-state probability distribution for the number of requests at arbitrary and at arrival time instants in PH/M/c-like systems with homogeneous servers in which the inter-arrival time distribution is represented by an acyclic set of memoryless phases. Our method is based on conditional probabilities and results in a simple computationally stable recurrence. It avoids the explicit manipulation of potentially large matrices and involves no iteration. Owing to the use of conditional probabilities, it delays the onset of numerical issues related to floating-point underflow as the number of servers and/or phases increases. For generalized Coxian distributions, the computational complexity of the proposed approach grows linearly with the number of phases in the distribution.

A Recurrent Solution of PH/M/c/N-Like and PH/M/c-Like Queues

We propose an efficient semi-numerical approach to compute the steady-state probability distribution for the number of requests at arbitrary and at arrival time instants in PH/M/c-like systems with homogeneous servers in which the inter-arrival time distribution is represented by an acyclic set of memoryless phases. Our method is based on conditional probabilities and results in a simple computationally stable recurrence. It avoids the explicit manipulation of potentially large matrices and involves no iteration. Owing to the use of conditional probabilities, it delays the onset of numerical issues related to floating-point underflow as the number of servers and/or phases increases. For generalized Coxian distributions, the computational complexity of the proposed approach grows linearly with the number of phases in the distribution.

Flexible servers in tandem lines with setup costs

We study the dynamic assignment of flexible servers to stations in the presence of setup costs that are incurred when servers move between stations. The goal is to maximize the long-run average profit. We provide a general problem formulation and some structural results, and then concentrate on tandem lines with two stations, two servers, and a finite buffer between the stations. We investigate how the optimal server assignment policy for such systems depends on the magnitude of the setup costs, as well as on the homogeneity of servers and tasks. More specifically, for systems with either homogeneous servers or homogeneous tasks, small buffer sizes, and constant setup cost, we prove the optimality of "multiple threshold" policies (where servers' movement between stations depends on both the number of jobs in the system and the locations of the servers) and determine the values of the thresholds. For systems with heterogeneous servers and tasks, small buffers, and constant setup cost, we provide results that partially characterize the optimal server assignment policy. Finally, for systems with larger buffer sizes and various service rate and setup cost configurations, we present structural results for the optimal policy and provide numerical results that strongly support the optimality of multiple threshold policies.