Software Bottlenecks Research Articles

A Systematic Method for Detecting Parallelized Software Bottlenecks and Suggesting Modifications: The Case of the Expectation Maximization Algorithm

A Systematic Method for Detecting Parallelized Software Bottlenecks and Suggesting Modifications: The Case of the Expectation Maximization Algorithm

A performance model for the communication in fast multipole methods on high-performance computing platforms

Exascale systems are predicted to have approximately 1 billion cores, assuming gigahertz cores. Limitations on affordable network topologies for distributed memory systems of such massive scale bring new challenges to the currently dominant parallel programing model. Currently, there are many efforts to evaluate the hardware and software bottlenecks of exascale designs. It is therefore of interest to model application performance and to understand what changes need to be made to ensure extrapolated scalability. The fast multipole method (FMM) was originally developed for accelerating N-body problems in astrophysics and molecular dynamics but has recently been extended to a wider range of problems. Its high arithmetic intensity combined with its linear complexity and asynchronous communication patterns make it a promising algorithm for exascale systems. In this paper, we discuss the challenges for FMM on current parallel computers and future exascale architectures, with a focus on internode communication. We focus on the communication part only; the efficiency of the computational kernels are beyond the scope of the present study. We develop a performance model that considers the communication patterns of the FMM and observe a good match between our model and the actual communication time on four high-performance computing (HPC) systems, when latency, bandwidth, network topology, and multicore penalties are all taken into account. To our knowledge, this is the first formal characterization of internode communication in FMM that validates the model against actual measurements of communication time. The ultimate communication model is predictive in an absolute sense; however, on complex systems, this objective is often out of reach or of a difficulty out of proportion to its benefit when there exists a simpler model that is inexpensive and sufficient to guide coding decisions leading to improved scaling. The current model provides such guidance.

Achieving One Billion Key-Value Requests per Second on a Single Server

Distributed in-memory key-value stores (KVSs) have become a critical data-serving layer in cloud computing and big data infrastructure. Unfortunately, KVSs have demonstrated a gap between achieved and available performance, QoS, and energy efficiency on commodity platforms. Two research thrusts have focused on improving key-value performance: hardware-centric research has started to explore specialized platforms for KVSs, and software-centric research revisited the KVS application to address fundamental software bottlenecks. Unlike prior research focusing on hardware or software in isolation, the authors aimed to full-stack (software through hardware) architect high-performance and efficient KVS platforms. Their full-system characterization identifies the critical hardware/software ingredients for high-performance KVS systems and suggests optimizations to achieve record-setting performance and energy efficiency: 120~167 million requests per second (RPS) on a single commodity server. They propose a future many-core platform and via detailed simulations demonstrate the capability of achieving a billion RPS with a single server platform.

Full-Stack Architecting to Achieve a Billion-Requests-Per-Second Throughput on a Single Key-Value Store Server Platform

Distributed in-memory key-value stores (KVSs), such as memcached, have become a critical data serving layer in modern Internet-oriented data center infrastructure. Their performance and efficiency directly affect the QoS of web services and the efficiency of data centers. Traditionally, these systems have had significant overheads from inefficient network processing, OS kernel involvement, and concurrency control. Two recent research thrusts have focused on improving key-value performance. Hardware-centric research has started to explore specialized platforms including FPGAs for KVSs; results demonstrated an order of magnitude increase in throughput and energy efficiency over stock memcached. Software-centric research revisited the KVS application to address fundamental software bottlenecks and to exploit the full potential of modern commodity hardware; these efforts also showed orders of magnitude improvement over stock memcached. We aim at architecting high-performance and efficient KVS platforms, and start with a rigorous architectural characterization across system stacks over a collection of representative KVS implementations. Our detailed full-system characterization not only identifies the critical hardware/software ingredients for high-performance KVS systems but also leads to guided optimizations atop a recent design to achieve a record-setting throughput of 120 million requests per second (MRPS) (167MRPS with client-side batching) on a single commodity server. Our system delivers the best performance and energy efficiency (RPS/watt) demonstrated to date over existing KVSs including the best-published FPGA-based and GPU-based claims. We craft a set of design principles for future platform architectures, and via detailed simulations demonstrate the capability of achieving a billion RPS with a single server constructed following our principles.

Architecting to achieve a billion requests per second throughput on a single key-value store server platform

Distributed in-memory key-value stores (KVSs), such as memcached, have become a critical data serving layer in modern Internet-oriented datacenter infrastructure. Their performance and efficiency directly affect the QoS of web services and the efficiency of datacenters. Traditionally, these systems have had significant overheads from inefficient network processing, OS kernel involvement, and concurrency control. Two recent research thrusts have focused upon improving key-value performance. Hardware-centric research has started to explore specialized platforms including FPGAs for KVSs; results demonstrated an order of magnitude increase in throughput and energy efficiency over stock memcached. Software-centric research revisited the KVS application to address fundamental software bottlenecks and to exploit the full potential of modern commodity hardware; these efforts too showed orders of magnitude improvement over stock memcached. We aim at architecting high performance and efficient KVS platforms, and start with a rigorous architectural characterization across system stacks over a collection of representative KVS implementations. Our detailed full-system characterization not only identifies the critical hardware/software ingredients for high-performance KVS systems, but also leads to guided optimizations atop a recent design to achieve a record-setting throughput of 120 million requests per second (MRPS) on a single commodity server. Our implementation delivers 9.2X the performance (RPS) and 2.8X the system energy efficiency (RPS/watt) of the best-published FPGA-based claims. We craft a set of design principles for future platform architectures, and via detailed simulations demonstrate the capability of achieving a billion RPS with a single server constructed following our principles.

H.264/AVC prediction modules complexity analysis

AbstractIn this paper we study the computational complexity of the intra/inter prediction modules for video coding software according to H.264/AVC standard. We analyse separately the complexity of each mode toward an implementation on TMS320C64 platform. Since any real‐time implementation is considered to be time and money consuming, complexity pre‐analysis of any giving video‐coding algorithm is very important to determine not only software bottlenecks, but also the minimum hardware resources and characteristics required for the targeted platform. To estimate the total complexity of the prediction processes (intra/inter), we assume that the encoder tests all modes. The total cycle's account is estimated by multiplying the number of cycles of each mode by the number of macroblocks in the frame. Simulation results indicate that our methodology for the complexity analyses for H.264/AVC prediction modules provides a good approximation with respect to the experimental results. Copyright © 2006 AEIT.

Performance of Scheduling Strategies for Client–Server Systems

Two approaches to the improvement of the performance of client–server systems, multithreading and scheduling of servers, are investigated. Both of these approaches are observed to have a significant impact on system performance. The use of multithreading improves throughput characteristics of systems, whereas the deployment of appropriate scheduling strategies at servers can produce a significant improvement in mean client response times. Based on a simulation model a number of basic questions that are important in the context of scheduling on nonmultithreaded, as well as multithreaded, systems are analyzed. Two important factors, monopolization of servers by large requests and software bottlenecks, are observed to be important in the context of scheduling on client–server systems. Both server scheduling, as well as multithreading, can be used to control these effects and lead to a higher system performance. Scheduling policies based on request characteristics are observed to perform well. A new request characteristic that is useful in the scheduling of client–server systems in the presence of software bottlenecks is proposed. Selection of both the server process, as well as the thread within the server, is required when multiple server, are co-located on the same CPU. A comparison between two scheduling approaches, single level and two level is presented in the paper. The results of this research are useful primarily in the design of operating systems for client–server systems and are also of interest to system designers and users.

System optimization for OLTP workloads

Major performance enhancements in large commercial systems are best achieved when advances in hardware technology are matched with advances in software technology. This article connects recent AS/400 hardware advances with the corresponding approaches used to tune the system performance for large online transaction processing (OLTP) workloads. We particularly emphasize those tuning efforts that affect the memory system. OLTP workloads are large and complex, stressing many parts of both the software and hardware. These workloads quickly expose software bottlenecks caused by contention on software locks. They also have large working sets, populated with hard-to-predict access patterns that make cache miss rates high. This causes the processor to spend a significant part of its execution time waiting for memory accesses. In multiprocessor systems, compilers alone have minimal effect on cycles spent in storage latency. Other optimizations are needed to affect this portion of the execution time, and many of those require direct involvement of the system software.

Software bottlenecking in client-server systems and rendezvous networks

Software bottlenecks are performance constraints caused by slow execution of a software task, in typical client-server systems a client task must wait in a blocked state for the server task to respond to its requests, so a saturated server will slow down all its clients. A rendezvous network generalizes this relationship to multiple layers of servers with send-and-wait interactions (rendezvous), a two-phase model of task behavior, and to a unified model for hardware and software contention. Software bottlenecks have different symptoms, different behavior when the system is altered, and a different cure from the conventional bottlenecks seen in queueing network models of computer systems, caused by hardware limits. The differences are due to the push-back effect of the rendezvous, which spreads the saturation of a server to its clients. The paper describes software bottlenecks by examples, gives a definition, shows how they can be located and alleviated, and gives a method for estimating the performance benefit to be obtained. Ultimately, if all the software bottlenecks can be removed, the performance limit will be due to a conventional hardware bottleneck. >

Recent Developments in Statistical Signal Processing

Publisher Summary Determining optimal algorithms and implementing the optimal algorithms are the two major aspects of statistical signal processing. There is considerable scope for application of ideas from linear system theory. One way to restrict complexity is to make further assumptions on the problem. A very common one in the last two decades has been to assume that there is an underlying finite-dimensional state-space model for the problem, in which case the memory can be fixed at the size of the state vector; however, the resulting filters will still be time variant. In any case, the important point is that these new results make for a much more direct link between signal processing theory and digital circuit implementations. This offers some promise in significantly reducing the software bottlenecks encountered in fitting all algorithms into simple general purpose architecture.