Multiprogramming Level Research Articles

Mean Field Analysis of Join-Below-Threshold Load Balancing for Resource Sharing Servers

Load balancing plays a crucial role in many large scale computer systems. Much prior work has focused on systems with First-Come- First-Served (FCFS) servers. However, servers in practical systems are more complicated. They serve multiple jobs at once, and their service rate can depend on the number of jobs in service. Motivated by this, we study load balancing for systems using Limited- Processor-Sharing (LPS). Our model has heterogeneous servers, meaning the service rate curve and multiprogramming level (limit on the number of jobs sharing the processor) differs between servers. We focus on a specific load balancing policy: Join-Below-Threshold (JBT), which associates a threshold with each server and, whenever possible, dispatches to a server which has fewer jobs than its threshold. Given this setup, we ask: how should we configure the system to optimize objectives such as mean response time? Configuring the system means choosing both a load balancing threshold and a multiprogramming level for each server. To make this question tractable, we study the many-server mean field regime.

Mean Field Analysis of Join-Below-Threshold Load Balancing for Resource Sharing Servers

Load balancing plays a crucial role in many large scale computer systems. Much prior work has focused on systems with First-Come-First-Served (FCFS) servers. However, servers in practical systems are more complicated. They serve multiple jobs at once, and their service rate can depend on the number of jobs in service. Motivated by this, we study load balancing for systems using Limited-Processor-Sharing (LPS). Our model has heterogeneous servers, meaning the service rate curve and multiprogramming level (limit on the number of jobs sharing the processor) differs between servers. We focus on a specific load balancing policy: Join-Below-Threshold (JBT), which associates a threshold with each server and, whenever possible, dispatches to a server which has fewer jobs than its threshold. Given this setup, we ask: how should we configure the system to optimize objectives such as mean response time? Configuring the system means choosing both a load balancing threshold and a multiprogramming level for each server. To make this question tractable, we study the many-server mean field regime. In this paper we provide a comprehensive study of JBT in the mean field regime. We begin by developing a mean field model for the case of exponentially distributed job sizes. The evolution of our model is described by a differential inclusion, which complicates its analysis. We prove that the sequence of stationary measures of the finite systems converges to the fixed point of the differential inclusion, provided a unique fixed point exists. We derive simple conditions on the service rate curves to guarantee the existence of a unique fixed point. We demonstrate that when these conditions are not satisfied, there may be multiple fixed points, meaning metastability may occur. Finally, we give a simple method for determining the optimal system configuration to minimize the mean response time and related metrics. While our theoretical results are proven for the special case of exponentially distributed job sizes, we provide evidence from simulation that the system becomes insensitive to the job size distribution in the mean field regime, suggesting our results are more generally applicable.

Response Time Distribution of a Class of Limited Processor Sharing Queues

Processor sharing queues are often used to study the performance of time-sharing systems. In such systems the total service rate ?(m) depends on the number of jobs m present in the system and there is a limit implemented, called the multi-programming level (MPL), on the number of jobs k that can be served simultaneously. Prior work showed that under highly variable jobs sizes, setting the MPL k beyond the value k = arg maxm ?(m) may reduce the mean response time. In order to study the impact of the MPL k on the response time distribution, we analyse the MAP/PH/LPSk( m) queue. In such a queue jobs arrive according to a Markovian arrival process (MAP), have phase-type (PH) distributed sizes, at most k jobs are processed in parallel and the total service rate depends on the number of jobs being served. Jobs that arrive when there are k or more jobs present are queued. We derive an expression for the Laplace transform of the response time distribution and numerically invert it to study the impact of the MPL k. Numerical results illustrate to what extent increasing k beyond k? increases the quantiles and tail probabilities of the response time distribution. They further demonstrate that for bursty arrivals and larger MPL k values having more variable job sizes may reduce the mean response time.

Analysis of queueing model with processor sharing discipline and customers impatience

Queueing systems with processor sharing represent the adequate models for sharing the resources, e.g., components of a computer or a bandwidth of communication systems. In this paper, we consider a queueing system with processor sharing discipline under quite general assumptions about the arrival and service processes. Arrivals are defined by the Markovian arrival process. The service time has a phase type distribution. Possible impatience of customers is taken into account. The number of customers, which can simultaneously obtain service, is limited. We compare two approaches for monitoring service of customers, namely, the approach counting the number of customers at each phase of service and the approach counting the phase of service of each customer and show the significant advantage of the former approach. We obtain the joint distribution of the number of customers in the system and the states of the underlying arrival and service processes as well as the loss probabilities. It is shown that the sojourn time in the system of an arbitrary customer has phase type distribution and an irreducible representation of this distribution is obtained. Numerical examples are presented. A possibility of optimal choice of the server capacity (e.g., multi-programming level) is numerically illustrated. An opportunity of increasing the speed of computations via the use of the graphics processing unit is discussed.

MaaT

The past decade has witnessed an increasing adoption of cloud database technology, which provides better scalability, availability, and fault-tolerance via transparent partitioning and replication, and automatic load balancing and fail-over. However, only a small number of cloud databases provide strong consistency guarantees for distributed transactions, despite decades of research on distributed transaction processing, due to practical challenges that arise in the cloud setting, where failures are the norm, and human administration is minimal. For example, dealing with locks left by transactions initiated by failed machines, and determining a multi-programming level that avoids thrashing without under-utilizing available resources, are some of the challenges that arise when using lock-based transaction processing mechanisms in the cloud context. Even in the case of optimistic concurrency control, most proposals in the literature deal with distributed validation but still require the database to acquire locks during two-phase commit when installing updates of a single transaction on multiple machines. Very little theoretical work has been done to entirely eliminate the need for locking in distributed transactions, including locks acquired during two-phase commit. In this paper, we re-design optimistic concurrency control to eliminate any need for locking even for atomic commitment, while handling the practical issues in earlier theoretical work related to this problem. We conduct an extensive experimental study to evaluate our approach against lock-based methods under various setups and workloads, and demonstrate that our approach provides many practical advantages in the cloud context.

Identifying the optimal level of parallelism in transactional memory applications

In this paper we investigate the issue of automatically identifying the "natural" degree of parallelism of an application using software transactional memory (STM), i.e., the workload-specific multiprogramming level that maximizes application's performance. We discuss the importance of adapting the concurrency level in two different scenarios, a shared-memory and a distributed STM infrastructure. We propose and evaluate two alternative self-tuning methodologies, explicitly tailored for the considered scenarios. In shared-memory STM, we show that lightweight, black-box approaches relying solely on on-line exploration can be extremely effective. For distributed STMs , we introduce a novel hybrid approach that combines model-driven performance forecasting techniques and on-line exploration in order to take the best of the two techniques, namely enhancing robustness despite model's inaccuracies, and maximizing convergence speed towards optimum solutions.

Tail-robust scheduling via limited processor sharing

From a rare events perspective, scheduling disciplines that work well under light (exponential) tailed workload distributions do not perform well under heavy (power) tailed workload distributions, and vice versa, leading to fundamental problems in designing schedulers that are robust to distributional assumptions on the job sizes. This paper shows how to exploit partial workload information (system load) to design a scheduler that provides robust performance across heavy-tailed and light-tailed workloads. Specifically, we derive new asymptotics for the tail of the stationary sojourn time under Limited Processor Sharing (LPS) scheduling for both heavy-tailed and light-tailed job size distributions, and show that LPS can be robust to the tail of the job size distribution if the multiprogramming level is chosen carefully as a function of the load.

Quantifying isolation anomalies

Choosing a weak isolation level such as Read Committed is understood as a trade-off, where less isolation means that higher performance is gained but there is an increased possibility that data integrity will be lost. Previously, one side of this trade-off has been carefully studied quantitatively -- there are well-known metrics for performance such as transactions per minute, standardized benchmarks that measure these in a controlled way, and analytic models that can predict how performance is influenced by system parameters like multiprogramming level. This paper contributes to quantifying the other aspect of the trade-off. We define a novel microbenchmark that measures how rapidly integrity violations are produced at different isolation levels, for a simple set of transactions. We explore how this rate is impacted by configuration factors such as multiprogramming level, or contention frequency. For the isolation levels in multi-version platforms (Snapshot Isolation and the multiversion variant of Read Committed), we offer a simple probabilistic model that predicts the rate of integrity violations in our microbenchmark from configuration parameters. We validate the predictive model against measurements from the microbenchmark. The model identifies a region of the configuration space where a surprising inversion occurs: for these parameter settings, more integrity violations happen with Snapshot Isolation than with multi-version Read Committed, even though the latter is considered a lower isolation level.

Performance Comparison of Coscheduling Algorithms for Non-Dedicated Clusters Through a Generic Framework

In this paper, we address several key issues in designing coscheduling algorithms for clusters. First, we propose a generic framework for deploying coscheduling techniques by providing a reusable and dynamically loadable kernel module. Second, we implement several communication-driven coscheduling algorithms [ dynamic coscheduling (DCS), spin block (SB) and periodic boost (PB)] on a 16- node Linux cluster using the above framework. Third, with exhaustive experimentation using mixed workloads, we observe that unlike PB, which provided the best performance on a Solaris platform, the SB scheme outperforms all other techniques on a Linux platform. Finally, we investigate the impact of several job placement strategies, multiprogramming level (MPL), communication intensity and CPU and I/O intensive jobs on the performance of these coscheduling schemes. The experimental results show that the blocking-based coscheduling scheme (SB) has better tolerance to system workload variation compared with the spin-based schemes (DCS and PB), and overall, the blocking-based coscheduling scheme seems a better choice for non-dedicated Linux clusters.

Graph-based Optimistic Transaction Management.

In this paper, we introduce and describe directed dependency graph-based transaction and concurrency control (DCC) for persistent (stable, single-level) object-based bulk data management systems. The new technique is optimistic and applicable across a wide range of store sizes, transaction sizes and multi-programming levels. It is also has potential for use in management of transactions in other contexts, for example web services.

Performance analysis of long-lived cooperative transactions in active DBMS

Active database management systems (ADBMS) are used in different application domains and especially for cooperative and long duration activity management. This paper deals with performance analysis of long-lived cooperative transaction processing in an ADBMS. We first briefly discuss NP-QuadLock – a concurrency control scheme for cooperative and long durational transactions in ADBMS. A restricted version of NP-QuadLock named 2L-QuadLock has been used for simulation. We have modeled such an ADBMS supporting 2L-QuadLock scheme by a queuing model. The failure of the transactions running in such systems has been modeled by a failure recovery model. We have simulated this model for a transaction processing system serving long-lived and cooperative transactions. We also discuss some important emerging application scenarios, where the proposed cooperative complex transaction mechanism can be used (e.g. 3G-service environment, ubiquitous computing environment, feature composition in intelligent network environment, multi-site and multi-domain web-services). An important objective of our work is to analyze quantitatively (a) the performance penalty on the system due to the partial abort, the number of locks held by a transaction, the number of states of the transactions, and (b) the gain in the performance of the system with the cooperation semantics proposed in 2L-QuadLock concurrency control mechanism. We have analyzed the effect of various parameters such as partial abort rate, cooperation rate, number of locks held by a transaction, multiprogramming level, on the performance metrics such as average service time, average saga length and the degree of compensation. Later, we characterize the application scenarios based on some important simulation parameters, and discuss the application performance needs for each of the application scenarios. The required performance parameters that need to be used for these application scenarios and the corresponding performance results using 2L-QuadLock are also discussed.

Adaptive time/space sharing with SCOJO

Time-shared execution of parallel jobs via gang scheduling is known to yield better average response times than space sharing. We incorporate adaptive CPU/node-resource allocation to consider varying system load and to reduce fragmentation. As main innovations, our SCOJO approach provides a clear model how to adapt, and considers realistic job mixes with moldable, malleable and rigid jobs. Our adaptive SCOJO significantly decreases response times and average slowdowns, while using a lower multiprogramming level than standard gang scheduling uses best and, therefore, decreasing the memory pressure. These benefits apply though the realistic job mixes limit the flexibility in resource allocation.

Performance-driven processor allocation

In current multiprogrammed multiprocessor systems, to take into account the performance of parallel applications is critical to decide an efficient processor allocation. In this paper, we present the performance-driven processor allocation policy (PDPA). PDPA is a new scheduling policy that implements a processor allocation policy and a multiprogramming-level policy, in a coordinated way, based on the measured application performance. With regard to the processor allocation, PDPA is a dynamic policy that allocates to applications the maximum number of processors to reach a given target efficiency. With regard to the multiprogramming level, PDPA allows the execution of a new application when free processors are available and the allocation of all the running applications is stable, or if some applications show bad performance. Results demonstrate that PDPA automatically adjusts the processor allocation of parallel applications to reach the specified target efficiency, and that it adjusts the multiprogramming level to the workload characteristics. PDPA is able to adjust the processor allocation and the multiprogramming level without human intervention, which is a desirable property for self-configurable systems, resulting in a better individual application response time.

TPF: a dynamic system thrashing protection facility

AbstractOperating system designers attempt to keep high CPU utilization by maintaining an optimal multiprogramming level (MPL). Although running more processes makes it less likely to leave the CPU idle, too many processes adversely incur serious memory competition, and even introduce thrashing, which eventually lowers CPU utilization. A common practice to address the problem is to lower the MPL with the aid of process swapping out/in operations. This approach is expensive and is only used when the system begins serious thrashing. The objective of our study is to provide highly responsive and cost‐effective thrashing protection by adaptively conducting priority page replacement in a timely manner. We have designed a dynamic system Thrashing Protection Facility (TPF) in the system kernel. Once TPF detects system thrashing, one of the active processes will be identified for protection. The identified process will have a short period of privilege in which it does not contribute its least recently used (LRU) pages for removal so that the process can quickly establish its working set, improving the CPU utilization. With the support of TPF, thrashing can be eliminated in its early stage by adaptive page replacement, so that process swapping will be avoided or delayed until it is truly necessary. We have implemented TPF in a current and representative Linux kernel running on an Intel Pentium machine. Compared with the original Linux page replacement, we showthat TPF consistently and significantly reduces page faults and the execution time of each individual job in several groups of interacting SPEC CPU2000 programs. We also show that TPF introduces little additional overhead to program executions, and its implementation in Linux (or Unix) systems is straightforward. Copyright © 2002 John Wiley & Sons, Ltd.

TPF: a dynamic system thrashing protection facility

Operating system designers attempt to keep high CPU utilization by maintaining an optimal multiprogramming level (MPL). Although running more processes makes it less likely to leave the CPU idle, too many processes adversely incur serious memory competition, and even introduce thrashing, which eventually lowers CPU utilization. A common practice to address the problem is to lower the MPL with the aid of process swapping out/in operations. This approach is expensive and is only used when the system begins serious thrashing. The objective of our study is to provide highly responsive and cost-effective thrashing protection by adaptively conducting priority page replacement in a timely manner. We have designed a dynamic system Thrashing Protection Facility (TPF) in the system kernel. Once TPF detects system thrashing, one of the active processes will be identified for protection. The identified process will have a short period of privilege in which it does not contribute its least recently used (LRU) pages for removal so that the process can quickly establish its working set, improving the CPU utilization. With the support of TPF, thrashing can be eliminated in its early stage by adaptive page replacement, so that process swapping will be avoided or delayed until it is truly necessary. We have implemented TPF in a current and representative Linux kernel running on an Intel Pentium machine. Compared with the original Linux page replacement, we show that TPF consistently and significantly reduces page faults and the execution time of each individual job in several groups of interacting SPEC CPU2000 programs. We also show that TPF introduces little additional overhead to program executions, and its implementation in Linux (or Unix) systems is straightforward.

The Cyclone Server Architecture: streamlining delivery of popular content

We propose a new web server architecture optimized for delivery of large, popular files. Delivery of such files currently pose a scalability problem for conventional content providers, which must devote server-side resources in direct proportion to the high multiprogramming level induced by a set of these connections. While use of scalable multicast may remedy this problem some day, multicast is rarely supported in today's wide-area infrastructure. Our approach alleviates many of the most serious scalability problems by developing new server-side mechanisms capable of managing a large set of TCP connections transporting the same content. The strategy we employ relies on the use of fast forward error correcting (FEC) codes to generate encodings of popular content, of which only a small sliding window is cached in memory at any time instant. The concurrent TCP connections then access the content only from this shared window, which is globally useful to all clients. Our method hinges on eliminating unscalable TCP retransmission buffers, as we can “retransmit” fresh encoding packets in lieu of the originals with no performance degradation and with no modifications to client TCP stacks. Ultimately, our Cyclone server capitalizes on concurrency to maximize sharing of state across different request threads while minimizing context switching, disk I/O, thrashing under high load and the cache memory footprint. In this paper, we describe the design and prototype implementation of our approach as a Linux kernel subsystem.

Overlay striping and optimal parallel I/O for modern applications

Disk array systems are rapidly becoming the secondary-storage media of choice for many emerging applications with large storage and high bandwidth requirements. Striping data across the disks of a disk array introduces significant performance benefits mainly because the effective transfer rate of the secondary storage is increased by a factor equal to the stripe width. However, the choice of the optimal stripe width is an open problem: no general formal analysis has been reported and intuition alone fails to provide good guidelines. As a result one may find occasionally contradictory recommendations in the literature. With this work we first contribute an analytical calculation of the optimal stripe width. Second, we recognize that the optimal stripe width is sensitive to the multiprogramming level, which is not known a priori and fluctuates with time. Thus, calculations of the optimal stripe width are, by themselves only, of little practical use. For this reason we propose a novel striping technique, called overlay striping, which allows objects to be retrieved using a number of alternative stripe widths. We provide the detailed algorithms for our overlay striping method and we study the associated storage overhead and performance improvements and we show that we can achieve near optimal performance for very wide ranges of the possible multiprogramming levels, while incurring small storage overheads.

Performance of multiversion and distributed two-phase locking concurrency control mechanisms in distributed database

In this paper, we have presented a detailed simulation study of a distributed multiversion and a distributed two-phase locking concurrency control mechanism (CCM). Our experiment concentrated on measuring the effect of message overhead, read:write ratios, data partitioning, and partial replication on the performance of these mechanisms. The effect of these parameters has not been investigated in any previous work. We simultated a blind-write model for two reasons: (a) all other works studied the behavior of multiversion CCMs under read-before-write model and observed a similar result, and (b) the performance of any multiversion CCM has not been studied under a blind-write model. A blind-write model is not unrealistic, and intutively the multiversion should provide much better performance. We observed that multiversion outperforms wound-wait (WW) in both partitioned and partially replicated databases. Multi version (MV) handles read-only and write-only transactions efficiently, and after a certain write percentage the throughput improves with this percentage. The message overhead progressively becomes less significant as the MPL (multiprogramming level) increases, indicating that in a heavily loaded system the throughput is least sensitive to message cost. We found that in the partially replicated case, 50% write does not show the lowest performance, as observed in the partitioned case.

Benefits of speedup knowledge in memory-constrained multiprocessor scheduling

Recently, research on scheduling for large-scale multiprocessor systems has begun to take into consideration the memory sizes of jobs. The approach used to ensure good performance without considering memory is to increase the multiprogramming level as the load increases. But if jobs require significant amounts of memory, then there will be a point at which the system can no longer fit additional jobs in memory. The scheduler will then have to carefully choose the way it allocates processors to jobs that are running in order to maximize performance. In this paper, we investigate the benefit of having speedup knowledge about individual jobs in making such a choice. We find that if memory sizes and speedup characteristics of jobs are uncorrelated, then there may be moderate benefits in having speedup information, but if large-sized jobs tend to have better speedup than small-sized ones, as might occur in real systems, then much more significant benefits can be obtained. We propose scheduling strategies that exploit speedup information to improve performance, and evaluate them under a variety of workloads.

Benefits of speedup knowledge in memory-constrained multiprocessor scheduling

Abstract Recently, research on scheduling for large-scale multiprocessor systems has begun to take into consideration the memory sizes of jobs. The approach used to ensure good performance without considering memory is to increase the multiprogramming level as the load increases. But if jobs require significant amounts of memory, then there will be a point at which the system can no longer fit additional jobs in memory. The scheduler will then have to carefully choose the way it allocates processors to jobs that are running in order to maximize performance. In this paper, we investigate the benefit of having speedup knowledge about individual jobs in making such a choice. We find that if memory sizes and speedup characteristics of jobs are uncorrelated, then there may be moderate benefits in having speedup information, but if large-sized jobs tend to have better speedup than small-sized ones, as might occur in real systems, then much more significant benefits can be obtained. We propose scheduling strategies that exploit speedup information to improve performance, and evaluate them under a variety of workloads.