Server Pooling Research Articles

Server pooling models for separate and bounded queues

The focus of this paper is on queueing models with arrivals from several sources, where the number of waiting spaces for each arrival source is finite. The main distinction from prior studies is that in our models waiting spaces are separate. Within this context, we examine the benefit of pooling, regardless of whether arrival rates are equal or different. We present a Quasi-Birth-and-Death (QBD) model to address the general case, with simplified versions tailored to specific scenarios. One practical application of the proposed models can be found in the loading and unloading processes at container terminals.We define a measure for stochasticity-related inefficiency, denoted relative interaction delay (RID), and analyze its behavior for the case of a single waiting space for each source. We show analytically that in the base model, the RID approximation is inversely proportional to the number of pooled queues. Numerical evaluations show an added benefit of pooling when arrival rates differ, observing a linear enhancement that is notably more pronounced.

Towards a lightweight task scheduling framework for cloud and edge platform

Mobile devices are becoming ubiquitous in our daily lives, but they have limited computational capacity. Thanks to the advancement in the network infrastructure, task offloading from resource-constrained devices to the near edge and the cloud becomes possible and advantageous. Complete task offloading is now possible to almost limitless computing resources of public cloud platforms. Generally, the edge computing resources support latency-sensitive applications with limited computing resources, while the cloud supports latency-tolerant applications. This paper proposes one lightweight task-scheduling framework from cloud service provider perspective, for applications using both cloud and edge platforms. Here, the challenge is using edge and cloud resources efficiently when necessary. Such decisions have to be made quickly, with a small management overhead. Our framework aims at solving two research questions. They are: (i) How to distribute tasks to the edge resource pools and multi-clouds? (ii) How to manage these resource pools effectively with low overheads? To answer these two questions, we examine the performance of our proposed framework based on Reliable Server Pooling (RSerPool). We have shown via simulations that RSerPool, with the correct usage and configuration of pool member selection policies, can accomplish the cloud/edge setup resource selection task with a small overhead.

An appointment scheduling policy for healthcare systems with parallel servers and pre-determined quality of service

The appointment scheduling problem is well-known in the literature. The use of appointment systems has been adopted widely in many different fields, including service industries and especially healthcare.This research focuses on healthcare systems where patients arrive according to pre-assigned appointments. We consider healthcare systems with several parallel servers, where a given sequence of patients, with randomly distributed service durations and a possibility of no-shows, is to be scheduled. The aim is to minimize the end of day and increase resource utilization while a minimal probability of each appointment starting on time (quality of service) is required.We formulated the problem using mathematical programing and developed a multi-server numerical-based (MSN) algorithm to solve it. We conducted some experimental runs and checked the impact of the problem parameters on the end of day, customers’ average waiting time and the percentage of customers that waited for service. We also show how server pooling improves the above system measures. Finally, once the appointments are set, we develop a methodology to determine the shift length so as to balance overtime costs (costs of overtime hours) against undertime costs (costs of regular, unused hours).

The Queuing-First Approach for Tail Management of Interactive Services

Managing high-percentile tail latencies is key to designing user-facing cloud services. Rare system hiccups or unusual code paths make some requests take $\boldsymbol{10\;\times -100\;\times}$10×-100× longer than the average. Prior work seeks to reduce tail latency by trying to address primarily root causes of slow requests. However, often the bulk of requests comprising the tail are not these rare slow-to-execute requests. Rather, due to head-of-line blocking, most of the tail comprises requests enqueued behind slow-to-execute requests. Under high disparity service distributions, queuing effects drastically magnify the impact of rare system hiccups and can result in high tail latencies even under modest load. We demonstrate that improving the queuing behavior of a system often yields greater benefit than mitigating the individual system hiccups that increase service time tails. We suggest two general directions to improve system queuing behavior–server pooling and common-case service acceleration–and discuss circumstances where each is most beneficial.

On the security of reliable server pooling systems

In order to cope with the requirements of availability-critical internet services, reliable server pooling (RSerPool) has been developed as the new IETF standard for a lightweight server redundancy and session failover framework. While the service and pool management performance of RSerPool had already been the topic of various research papers, its security has not yet been widely examined. But security for availability-critical systems is crucial, since service outages – regardless of whether being caused by system failures or intentional denial of service (DoS) attacks – are not acceptable for the users of such systems. In this article, we first introduce RSerPool as well as the underlying SCTP protocol. In an analysis of the attack threats, we will show the possibilities of an attacker to degrade the service provided by an RSerPool system. We will furthermore introduce possible countermeasures, in order to prevent attacks and improve the robustness of the systems. We will finally show the effectiveness of our proposed countermeasures using simulations. In order to validate our simulation results, we furthermore compare them to measurements from a real-world internet setup using the PlanetLab.

Maximum Availability Server Selection Policy for Efficient and Reliable Session Control Systems

There has been a rapid growth of services based on session control. Session-based services comprise multimedia conferences, Internet telephone calls, instant messaging, and similar applications consisting of one or more media types such as audio and video. Deployment examples include session control services as part of the IP multimedia subsystem (IMS), in the third-generation mobile networks. High service dependability in session control systems is achieved by introducing redundancy, e.g., through reliable server pooling (RSerPool) or clustering. Namely, session control servers are multiplied in server sets. Performance of such replicated session control servers is quantified by transaction control time. Thus, reducing transaction control time enhances performance. Server selection policies (SSP) are crucial in achieving this goal. The maximum availability (MA) SSP is proposed to improve session control performance in scenarios with server and communication failures. Based on a status vector, MA aims at maximizing the probability of successful transaction with the current transmission, thereby minimizing the average number of attempted servers until success. MA is applicable in a broad range of IP-based systems and services, and it is independent of the fault-tolerant platform. A simple protocol extension is proposed in order to integrate MA into the RSerPool fault-tolerant architecture. In addition, an analytic model is derived based on certain system model assumptions. Analytic and simulation results show that transaction control time is considerably reduced with MA as opposed to when using traditional round robin

Securing reliable server pooling in MANET against byzantine adversaries

Reliable server pooling (rSerPool) is an architecture and a set of protocols allowing a service provider to run several servers that can reliably provide the same service. Should a particular server fail while providing its service, another server can efficiently replace it. This property is attractive not only for wired but also for wireless networks. However, the unique characteristics of mobile ad hoc networks (MANETs) bring serious reliability and security challenges to the application of rSerPool. In this paper, we perform a comprehensive investigation of the security of rSerPool in MANET against both server failures and, especially, Byzantine attacks. We formulate security requirements for rSerPool in MANET and design efficient, distributed, and survivable security solutions for both main phases of rSerPool: service discovery and service provision. Specifically, we secure the service discovery phase by using a secure multiple-dominating set creation protocol, and the service provision phase by using a novel type of threshold signature scheme. Both protocols address novel security goals and are of independent interest as they can find applications to other areas; most notably, the construction of a distributed and survivable public-key infrastructure in MANET.

Evaluation of Architectures for Reliable Server Pooling in Wired and Wireless Environments

Reliable server pooling (RSP) allows a pool of redundant information sources to be viewed as a single transport endpoint and, therefore, it is able to provide persistent connections and balanced traffic for different applications. The Internet Engineering Task Force RSerPool Working Group has proposed an architecture to implement the RSP; it defines an overlay network providing an upper layer protocol or an application with a range of reliability services, from simple server selection to a fully automatic session-failover capability. The simulation experiments conducted in both wired and wireless environments show that the current version of the RSerPool works well in fixed and relatively reliable environments, but its performance worsens rapidly as the networks become more unreliable or mobile. The issues we have identified in wireless mobile ad hoc networks include network partition, high signaling overhead, difficulty in synchronization among name servers, and excessive aggressiveness in handling failures. Alternative design options for the RSP in wireless and mobile environments are introduced and evaluated.

Optimal Workload Allocation in Open Networks of Multiserver Queues

In this paper, we examine the general problem of workload allocation in an open Jackson network of multiserver queues. We show that use of the open network model leads to a separable, convex formulation of the problem with relatively simple optimality conditions. Using these conditions, we prove in general that server groups with the same number of servers should be loaded equally and larger groups should be loaded more heavily than smaller groups. It is also shown that server pooling, combining servers into larger groups, will always reduce congestion and/or increase throughput. We discuss the significance of our results for job shop applications and also for the concept of a production bottleneck. In systems with an unbalanced configuration of servers, traditional, deterministic bottleneck analysis is distinctly nonoptimal; our results provide a simple way to locate production bottlenecks in shops with significant queueing effects.