IBM Power Research Articles

Communication protocol optimization for enhanced GPU performance

The U.S. Department of Energy CORAL program systems SUMMIT and SIERRA are based on hybrid servers comprising IBM POWER9 CPUs and NVIDIA V100 graphics processing units (GPUs) connected by two extended data rate (EDR) links to a high-speed InfiniBand Network. A major challenge to the communication software stack is to optimize performance for all combinations of data origin and destination: host or GPU memory, same or different server. Alternate paths exist for routing data from GPU memory. When origin and destination are on different servers, it can be sent either via host memory or bypassing host memory with GPU direct feature. When origin and destination are on the same server, host memory can be bypassed with peer-to-peer inter process communication (IPC). For large messages pipelining makes host memory data path competitive with GPU direct. In this article, we explain the techniques used in Spectrum MPI Parallel Active Message Interface layer to cache memory types and attributes in order to reduce the overhead associated with calling the CUDA application programming interface (API); in addition, we detail the different protocols used for different memory types, device memory, managed memory, and host memory. To illustrate, the caching technique achieved a device-to-device latency improvement of 26% for intranode transfers and 19% for internode transfers.

Pre-exascale accelerated application development: The ORNL Summit experience

High-performance computing (HPC) increasingly relies on heterogeneous architectures to achieve higher performance. In the Oak Ridge Leadership Facility (OLCF), Oak Ridge, TN, USA, this trend continues as its latest supercomputer, Summit, entered production in early 2019. The combination of IBM POWER9 CPU and NVIDIA V100 GPU, along with a fast NVLink2 interconnect and other latest technologies, pushes system performance to a new height and breaks the exascale barrier by certain measures. Due to Summit's powerful GPUs and much higher GPU–CPU ratio, offloading to accelerators becomes a requirement for any application, which intends to effectively use the system. To facilitate navigating a complex landscape of competing heterogeneous architectures, a collection of applications from a wide spectrum of scientific domains is selected for early adoption on Summit. In this article, the experience and lessons learned are summarized, in the hope of providing useful guidance to address new programming challenges, such as scalability, performance portability, and software maintainability, for future application development efforts on heterogeneous HPC systems.

The high-speed networks of the Summit and Sierra supercomputers

Oak Ridge National Laboratory's Summit supercomputer and Lawrence Livermore National Laboratory's Sierra supercomputer utilize InfiniBand interconnect in a Fat-tree network topology, interconnecting all compute nodes, storage nodes, administration, and management nodes into one linearly scalable network. These networks are based on Mellanox 100-Gb/s EDR InfiniBand ConnectX-5 adapters and Switch-IB2 switches, with compute-rack packaging and cooling contributions from IBM. These devices support in-network computing acceleration engines such as Mellanox Scalable Hierarchical Aggregation and Reduction Protocol, graphics processor unit (GPU) Direct RDMA, advanced adaptive routing, Quality of Service, and other network and application acceleration. The overall IBM Spectrum Message Passing Interface (MPI) messaging software stack implements Open MPI, and was a collaboration between IBM, Mellanox, and NVIDIA to optimize direct communication between endpoints, whether compute nodes (with IBM POWER CPUs, NVIDIA GPUs, and flash memory devices), or POWER-hosted storage nodes. The Fat-tree network can isolate traffic among the compute partitions and to/from the storage subsystem, providing more predictable application performance. In addition, the high level of redundancy of this network and its reconfiguration capability ensures reliable high performance even after network component failures. This article details the hardware and software architecture and performance of the networks and describes a number of the high-performance computing (HPC) enhancements engineered into this generation of InfiniBand.

Redefining IBM power system design for CORAL

Stipulations in the 2014 Collaboration of Oak Ridge, Argonne, and Livermore (CORAL) joint procurement activity not only motivated a fundamental change in IBM's high-performance computer design, which refocused IBM power systems on compute nodes that can scale to 200 petaflops with access to 2.5 PB of memory, but also served the commercial market for single-server applications. The distribution of both processing elements and memory required a careful look at data movement. The resultant AC922 POWER9 system features NVIDIA V100 GPUs with cache line access granularity, more than double the IO bandwidth of PCIe Gen3, and low-latency interfaces interconnected by the state-of-the-art dual-rail Mellanox CAPI EDR HCAs running at 50 Gb/s. With processing units designed to operate at 250 and 300 W, a single system can produce up to 3,080 kW. The overall CORAL solutions achieved power usage effectiveness rankings in the top ten on the Green500. Previous power designs used uniquely designed cabinets and scaled-up infrastructure to achieve efficiency. For successful commercial use, our design uses industry-standard 19-in drawers and racks. Both air- and water-cooled solutions allow for use in a wide range of customer environments. This article documents the novel design features that facilitate data movement and enable new coherent programming models. It describes how three generations of system designs became the foundation for the CORAL contract fulfillment and illustrates key features and specifications of the final product.

Accelerating next generation sequencing data analysis: an evaluation of optimized best practices for Genome Analysis Toolkit algorithms.

Advancements in next generation sequencing (NGS) technologies have significantly increased the translational use of genomics data in the medical field as well as the demand for computational infrastructure capable processing that data. To enhance the current understanding of software and hardware used to compute large scale human genomic datasets (NGS), the performance and accuracy of optimized versions of GATK algorithms, including Parabricks and Sentieon, were compared to the results of the original application (GATK V4.1.0, Intel x86 CPUs). Parabricks was able to process a 50× whole-genome sequencing library in under 3 h and Sentieon finished in under 8 h, whereas GATK v4.1.0 needed nearly 24 h. These results were achieved while maintaining greater than 99% accuracy and precision compared to stock GATK. Sentieon’s somatic pipeline achieved similar results greater than 99%. Additionally, the IBM POWER9 CPU performed well on bioinformatic workloads when tested with 10 different tools for alignment/mapping.

Thread Isolation to Improve Symbiotic Scheduling on SMT Multicore Processors

Resource sharing is a critical issue in simultaneous multithreading (SMT) processors as threads running simultaneously on an SMT core compete for shared resources. Symbiotic job scheduling, which co-schedules applications with complementary resource demands, is an effective solution to maximize hardware utilization and improve overall system performance. However, symbiotic job scheduling typically distributes threads evenly among cores, i.e., all cores get assigned the same number of threads, which we find to lead to sub-optimal performance. In this paper, we show that asymmetric schedules (i.e., schedules that assign a different number of threads to each SMT core) can significantly improve performance compared to symmetric schedules. To leverage this finding, we propose thread isolation , a technique that turns symmetric schedules into asymmetric ones yielding higher overall system performance. Thread isolation identifies SMT-adverse applications and schedules them in isolation on a dedicated core to mitigate their sharp performance degradation under SMT. Our experimental results on an IBM POWER8 processor show that thread isolation improves system throughput by up to 5.5 percent compared to a state-of-the-art symmetric symbiotic job scheduler.

Communication Profiling and Characterization of Deep-Learning Workloads on Clusters With High-Performance Interconnects

Heterogeneous high-performance computing systems with GPUs are equipped with high-performance interconnects like InfiniBand, Omni-Path, PCIe, and NVLink. However, little exists in the literature that captures the performance impact of these interconnects on distributed deep learning (DL). In this article, we choose Horovod, a distributed training middleware, to analyze and profile various DNN training workloads using TensorFlow and PyTorch in addition to standard MPI microbenchmarks. We use a wide variety of systems with CPUs like Intel Xeon and IBM POWER9, GPUs like Volta V100, and various interconnects to analyze the following metrics: 1) message-size with Horovod's tensor-fusion; 2) message-size without tensor-fusion; 3) number of MPI/NCCL calls; and 4) time taken by each MPI/NCCL call. We observed extreme performance variations for non-power-of-two message sizes on different platforms. To address this, we design a message-padding scheme for Horovod, illustrate significantly smoother allreduce latency profiles, and report cases where we observed improvement for end-to-end training.

JUNGFRAU detector for brighter x-ray sources: Solutions for IT and data science challenges in macromolecular crystallography.

In this paper, we present a data workflow developed to operate the adJUstiNg Gain detector FoR the Aramis User station (JUNGFRAU) adaptive gain charge integrating pixel-array detectors at macromolecular crystallography beamlines. We summarize current achievements for operating at 9 GB/s data-rate a JUNGFRAU with 4 Mpixel at 1.1 kHz frame-rate and preparations to operate at 46 GB/s data-rate a JUNGFRAU with 10 Mpixel at 2.2 kHz in the future. In this context, we highlight the challenges for computer architecture and how these challenges can be addressed with innovative hardware including IBM POWER9 servers and field-programmable gate arrays. We discuss also data science challenges, showing the effect of rounding and lossy compression schemes on the MX JUNGFRAU detector images.

Vectorizing programs with IF-statements for processors with SIMD extensions

Vectorization of programs is crucial for achieving high performance on modern processors with SIMD (Single Instruction Multiple Data) extensions. Programs with IF-statements suffer from control flow divergence that seriously complicates automatic vectorization. Therefore, contemporary compilers employ the IF-conversion approach to convert control flow to data flow, which relies on using predicated execution techniques (i.e., masked or select SIMD instructions). In this paper, we enhance the compiler’s capabilities to generate efficiently vectorized code for processors without masked instructions. We improve the state of the art in program vectorization by developing a novel approach—IF-select transformation—which is applicable to arbitrarily nested IF-statements. We implement our approach in the open-source Open64 compiler and evaluate its performance on the SW26010 processor used in the Sunway TaihuLight supercomputer (currently #3 in the TOP500 list) that does not support masked instructions. We extend our vectorization approach by providing an additional LLVM optimization pass to reduce the amount of masked memory accesses on processors without masked instructions, e.g., IBM Power8 and ARMCortex-A8. Experimental results demonstrate the performance advantages of the suggested vectorization techniques.

Performance Evaluation of a Hybrid Computer Cluster Built on IBM POWER8 Microprocessors

This paper is devoted to the performance evaluation of a hybrid computer cluster built on IBM POWER8 CPUs and NVIDIA Tesla P100 GPUs. The architecture of the computing system and software used are described. Results of experiments carried out using the STREAM, NPB, Crossroads/NERSC-9 DGEMM, and HPL packages are discussed. The efficiency of the simultaneous multithreading (SMT) technology supported by POWER8 processors, as well as the performance of some compilers, parallel programming and mathematical libraries, on this architecture is analyzed.

Heterogeneity aware power abstractions for dynamic power dominated FinFET‐based microprocessors

In dynamic power dominated FinFET-based microprocessors, there is significant heterogeneity in the chip power profile induced due to various factors. The workloads the microprocessors operate on are inherently heterogeneous in their switching characteristics. There is also variation in power across the chip, even within the IP block for a given workload. The aim is to provide a comprehensive industry perspective on analysing and mitigating the problems and challenges posed by this heterogeneity in dynamic power signatures in the design of next generation 14 nm FinFET-based microprocessors like IBM POWER9. This broader view of design principles from a modern-day microprocessor like POWER9 is useful for the adoption of the techniques presented and further fuel-related research in the area of dynamic power management of FinFET-based microprocessor designs. Additional focus in the context of clock-gating techniques and other use cases is also placed. New approaches for heterogeneity-aware per clock-gating domain parameterised power abstractions for enabling rapid hierarchical chip power analysis are presented.

Power Delivery Network Modeling and Benchmarking for Emerging Heterogeneous Integration Technologies

In this article, a power delivery network (PDN) modeling framework for heterogeneous integration platforms is presented, which includes both IR-drop and transient analysis. The model is validated using IBM power grid benchmarks, and maximum relative errors are less than 7.3%. To evaluate interposer and bridge-chip-based integration platforms, we assume a field-programmable gate array (FPGA)-CPU 2.5-D integration platform, of which the FPGA consumes 45 W and the CPU consumes 75 W. The simulation results show that an interposer with dense power/ground grid and microbumps can suppress power supply noise (PSN) by a small margin with the requirement of high-density through-silicon vias (TSVs). For bridge-chip-based integration, under the assumption that the active dice above the bridge-chips are not connected to the package power/ground planes, some PDN challenges are evaluated. Using multiple bridge-chips and smaller overlap areas between the bridge-chips and the active dice, the worst-case PSN in bridge-chip-based integration is minimized. Next, we perform parametric studies for these 2.5-D integration platforms as a function of C4 bump pitch and metal layers. Lastly, we propose to use TSVs in the bridge-chip to reduce PSN. With nine bundled TSVs of $8~\mu \text{m}$ diameter, the IR-drop of bridge-chip-based integration technologies is only 1% higher than the standalone case.

PLASMA

The recent version of the Parallel Linear Algebra Software for Multicore Architectures (PLASMA) library is based on tasks with dependencies from the OpenMP standard. The main functionality of the library is presented. Extensive benchmarks are targeted on three recent multicore and manycore architectures, namely, an Intel Xeon, Intel Xeon Phi, and IBM POWER 8 processors.

EM-Aware and Lifetime-Constrained Optimization for Multisegment Power Grid Networks

This paper proposes a new power-ground (P/G) network sizing technique based on the recently proposed fast electromigration (EM) immortality check method for general multisegment interconnect wires and a new physics-based EM assessment technique for more accurate time to failure analysis. This paper first shows that the new P/G optimization problem, subject to the voltage IR drop and new EM constraints, can still be formulated as an efficient sequence of linear programing problem, where the optimization is carried out in two linear programing phases in each iteration. The new optimization will ensure that none of the wires fail if all the constraints are satisfied. However, requiring all the wires to be EM immortal can be overconstrained. To mitigate this problem, the first improvement is by means of adding reservoir branches to the mortal wires whose lifetime cannot be made immortal by wire sizing. This is a very effective approach as long as there is a sufficient reservoir area. The second improvement is to consider the aging effects of interconnect wires in the P/G networks. The idea is to allow some short-lifetime wires to fail and optimize the rest of the wires while considering the additional resistance caused by the failed wire segments. In this way, the resulting P/G networks can be optimized, such that the target lifetime of the whole P/G networks can be ensured and will become more robust and aging-aware over the expected lifetime of the chip. Numerical results on a number of IBM and self-generated power supply networks demonstrate that the new method can effectively reduce the area of the networks while ensuring immortality or enforcing target lifetime for all the wires, which is not the case for the existing current-density-constrained optimization methods.

ASAP: <underline>A</underline>ccelerated <underline>S</underline>hort-Read <underline>A</underline>lignment on <underline>P</underline>rogrammable Hardware

The proliferation of high-throughput sequencing machines ensures rapid generation of up to billions of short nucleotide fragments in a short period of time. This massive amount of sequence data can quickly overwhelm today's storage and compute infrastructure. This paper explores the use of hardware acceleration to significantly improve the runtime of short-read alignment, a crucial step in preprocessing sequenced genomes. We focus on the Levenshtein distance (edit-distance) computation kernel and propose the ASAP accelerator, which utilizes the intrinsic delay of circuits for edit-distance computation elements as a proxy for computation. Our design is implemented on an Xilinx Virtex 7 FPGA in an IBM POWER8 system that uses the CAPI interface for cache coherence across the CPU and FPGA. Our design is $200\times$ faster than the equivalent C implementation of the kernel running on the host processor and $2.2\times$ faster for an end-to-end alignment tool for 120-150 base-pair short-read sequences. Further the design represents a $3760\times$ improvement over the CPU in performance/Watt terms.

HPSM: a programming framework to exploit multi-CPU and multi-GPU systems simultaneously

This paper presents a high-level C++ framework to explore multi-CPU and multi-GPU systems called HPSM. HPSM enables execution of parallel loops and reductions simultaneously over CPUs and GPUs using three parallel backends: Serial, OpenMP, and StarPU. We analysed HPSM development effort with AXPY program through two standard metrics (NCLOC and ES). In addition, we evaluated performance and energy with three parallel benchmarks: N-Body, Hotspot, and CFD solver. HPSM reduced code effort by up to 56.9% compared to StarPU C interface, although it resulted in 2.5× more lines of code compared to OpenMP. The CPU-GPU combination attained speedup results with Hotspot of up to 92.7× on a X86-based system with four GPUs and up to 108.2× on an IBM POWER8+ system with two GPUs. On both systems, the addition of GPUs improved energy efficiency.

HPSM: a programming framework to exploit multi-CPU and multi-GPU systems simultaneously

This paper presents a high-level C++ framework to explore multi-CPU and multi-GPU systems called HPSM. HPSM enables execution of parallel loops and reductions simultaneously over CPUs and GPUs using three parallel backends: Serial, OpenMP, and StarPU. We analysed HPSM development effort with AXPY program through two standard metrics (NCLOC and ES). In addition, we evaluated performance and energy with three parallel benchmarks: N-Body, Hotspot, and CFD solver. HPSM reduced code effort by up to 56.9% compared to StarPU C interface, although it resulted in 2.5× more lines of code compared to OpenMP. The CPU-GPU combination attained speedup results with Hotspot of up to 92.7× on a X86-based system with four GPUs and up to 108.2× on an IBM POWER8+ system with two GPUs. On both systems, the addition of GPUs improved energy efficiency.

Exploiting Algorithmic Noise Tolerance for Scalable On-Chip Voltage Regulation

With the advent of on-chip digital low-dropout (DLDO) regulators, distributed on-chip voltage regulation has become increasingly promising. Environmental and operating conditions have been demonstrated to degrade DLDO performance, which directly affects execution accuracy. The area overhead (OH) needed to compensate aging-induced voltage noise degradation can be significant. Accordingly, in this paper, the algorithmic noise tolerance of certain processor components is exploited as an area–quality control knob to trade the program output quality for area OH. Furthermore, efficient and lightweight techniques utilizing a unidirectional shift register and reduced clock pulsewidth triggering are proposed to realize a novel aging-aware (AA) DLDO to achieve a better area and quality tradeoff. Owing to the large number and distributed nature of voltage regulators, with the proposed design, both the number of regulators utilized in the system and the size of each local regulator are scalable to satisfy the needs of different applications and processor components with varying algorithmic noise tolerance. It is demonstrated through simulation of an IBM POWER8 like processor that the proposed AA design can achieve up to, respectively, 43.2% and $3\times $ transient and steady-state performance improvement. Additionally, more than 10% area OH saving can be achieved over a 5-year period.

Electromigration Check: Where the Design and Reliability Methodologies Meet

Due to technology scaling, electromigration (EM) signoff has become increasingly difficult, mainly due to the use of inaccurate methods for EM assessment, such as the empirical Black’s model. Results of recent measurements performed on power grid-like structures isolated in the power grid environment have demonstrated that the weak link approach cannot accurately predict lifetime of the power grids. It calls for significant over-design, while, today, there is very little margin left for EM. Numerous observations clearly indicate that there is a need for a new EM checking approach that accurately models EM degradation using physics-based models, combined with a mesh model to account for redundancy, while being fast enough to be practically useful. In this paper, we present a novel approach for power grid EM checking using physics-based models that can account for process, voltage and temperature variations across the die. Existing physical models for EM in metal branches were extended to track EM degradation in multi-branch interconnect trees. Our results, for a number of IBM power grid benchmarks, confirm that Black’s model is overly inaccurate. The lifetimes found using our physics-based approach are on average 2.75x longer than those based on a Black’s model, as extended to handle mesh power grids. With a maximum runtime of 2.3 h among all the IBM benchmarks, our method appears to be suitable for large VLSI circuits.

Developing Efficient Implementations of Bellman–Ford and Forward-Backward Graph Algorithms for NEC SX-ACE

The main goal of this work is to demonstrate that the development of data-intensive appli- cations for vector systems is not only important and interesting, but is also very possible. In this paper we describe possible implementations of two fundamental graph-processing algorithms for an NEC SX-ACE vector computer: the Bellman–Ford algorithm for single source shortest paths computation and the Forward-Backward algorithm for strongly connected components detection. The proposed implementations have been developed and optimised in accordance with features and properties of the target architecture, which allowed them to achieve performance comparable to other traditional platforms, such as Intel Skylake, Intel Knight Landing or IBM Power processors.