IBM POWER8 Processor Research Articles

Using hardware-transactional-memory support to implement speculative task execution

Loops take up most of the time of computer programs, so optimizing them so that they run in the shortest time possible is a continuous task. However, this task is not negligible; on the contrary, it is an open area of research since many irregular loops are hard to parallelize. Generally, these loops have loop-carried (DOACROSS) dependencies and the appearance of dependencies could depend on the context. Many techniques have been studied to be able to parallelize these loops efficiently; however, for example in the OpenMP standard there is no efficient way to parallelize them. This article presents Speculative Task Execution (STE), a technique that enables the execution of OpenMP tasks in a speculative way to accelerate certain hot-code regions (such as loops) marked by OpenMP directives. It also presents a detailed analysis of the application of Hardware Transactional Memory (HTM) support for executing tasks speculatively and describes a careful evaluation of the implementation of STE using HTM on modern machines. In particular, we consider the scenario in which speculative tasks are generated by the OpenMP taskloop construct (Speculative Taskloop (STL)). As a result, it provides evidence to support several important claims about the performance of STE over HTM in modern processor architectures. Experimental results reveal that: (a) by implementing STL on top of HTM for hot-code regions, speed-ups of up to 5.39× can be obtained in IBM POWER8 and of up to 2.41× in Intel processors using 4 cores; and (b) STL-ROT, a variant of STL using rollback-only transactions (ROTs), achieves speed-ups of up to 17.70× in IBM POWER9 processor using 20 cores.

Thread Isolation to Improve Symbiotic Scheduling on SMT Multicore Processors

Resource sharing is a critical issue in simultaneous multithreading (SMT) processors as threads running simultaneously on an SMT core compete for shared resources. Symbiotic job scheduling, which co-schedules applications with complementary resource demands, is an effective solution to maximize hardware utilization and improve overall system performance. However, symbiotic job scheduling typically distributes threads evenly among cores, i.e., all cores get assigned the same number of threads, which we find to lead to sub-optimal performance. In this paper, we show that asymmetric schedules (i.e., schedules that assign a different number of threads to each SMT core) can significantly improve performance compared to symmetric schedules. To leverage this finding, we propose thread isolation , a technique that turns symmetric schedules into asymmetric ones yielding higher overall system performance. Thread isolation identifies SMT-adverse applications and schedules them in isolation on a dedicated core to mitigate their sharp performance degradation under SMT. Our experimental results on an IBM POWER8 processor show that thread isolation improves system throughput by up to 5.5 percent compared to a state-of-the-art symmetric symbiotic job scheduler.

Functionality and performance of NVLink with IBM POWER9 processors

Heterogeneous computer systems with multiple types of processing elements (PEs) are becoming a popular design to optimize performance and efficiency for a wide variety of applications. Each part of an application can be executed on the PE for which it is best suited. In heterogeneous systems, communication, efficient data movement, and memory sharing across PEs are critical to execute an application across the different PEs while incurring minimal overhead for communication and synchronization. The IBM POWER9 processor supports the NVIDIA NVLink interface, a high-performance interconnect with many such capabilities. In the IBM Power System AC922, IBM POWER9 processors directly connect to multiple NVIDIA GPUs using NVLink. In this paper, we highlight the important functional and performance capabilities of NVLink with the POWER9 processor. These include high bandwidth, hardware cache coherence, fine-grained data movement, and hardware support for atomic operations across all PEs of a compute node. We also present an analysis of how these performance and functional capabilities of POWER9 processors and NVLink are expected to have significant impacts on performance and programmability across a variety of important applications, such as machine learning and domains within high-performance computing.

IBM POWER9 processor core

The IBM POWER9 processor is the latest Reduced Instruction Set Computer microprocessor from IBM. POWER9 employs a new modular core microarchitecture to counter the technology trend of decreasing frequency and increasing power density from generation to generation. The new POWER9 design enables a family of processors optimized for a broad range of server applications. The new microarchitecture is closely coupled with a rich set of new instructions geared toward data-centric applications. In this paper, we describe the POWER9 core microarchitecture innovations, its new instructions and features, and the exploitation of this new design for computing in the cognitive era.

IBM POWER9 processor and system features for computing in the cognitive era

IBM POWER9 is a family of processor chips designed to serve a diverse set of workloads. New features have been added to POWER9 to address emerging workloads such as cognitive and artificial intelligence applications. POWER9 also further enhances features introduced in IBM POWER8 for big data and cloud applications. Distinct chips using common intellectual property building blocks are provided to enable enterprise applications requiring large symmetric multiprocessor servers with large memory footprints, as well as one to two socket industry form-factor servers. In this paper, we describe new POWER9 features for both system types. Several highly differentiated new features are described in other papers in this issue of the IBM Journal, and they provide a more in-depth description of their unique design aspects.

XIVE: External interrupt virtualization for the cloud infrastructure

In today's microprocessors, external interrupts are the only processor resource that has no full virtualization hardware support. Therefore, they create significant overhead in the form of virtual machine (VM) context switches, especially in cloud and hyperscale datacenter systems, in which the physical processors are shared by a large number of virtual processors allocated to the VMs running on the system. The IBM POWER9 processor closes this gap in the processor hardware virtualization support by introducing an eXternal Interrupt Virtualization Engine (XIVE) architecture. XIVE defines a holistic interrupt delivery mechanism that supports multiple layers of interrupt coalescing and hardware-based routing of interrupts to the correct physical thread and target level. As required, individual interrupts can thus be routed directly to a user process, to a specific supervisor or an operating system, or to the hypervisor, thereby eliminating the need for interrupt rerouting in software and minimizing the number of context switches. In addition, XIVE provides the means to automatically trigger escalation interrupts to the next higher privilege level in case the target of an interrupt is not dispatched. In this paper, we discuss the fundamental concepts of the XIVE architecture and provide an overview of the XIVE-based interrupt controller implementation of the IBM POWER9 processor.

IBM POWER9 package technology and design

The first-level package that contains the IBM POWER9 processor chip is designed to achieve the high computational performance needed for cognitive systems in a cost-effective design. The throughput data bandwidth of the POWER9 package for high-end scale-up systems is more than 1 TB/s, which is double the data bandwidth of the previous generation. This increase in bandwidth is achieved by introducing a dielectric with a loss tangent of 40% of the predecessor material, a C4 density increase of 15%, higher number of stacked vias to reduce jogging, and improved via pattern and placement to increase the frequency and density of signals. The cloud platform scale-out POWER9 package leverages the high-end and cognitive platform package attributes to maintain signal frequency while introducing novel chip-package-system co-design techniques. These design techniques were used to produce a well-balanced two-socket entry-level package with four build-up layers above and below the core, instead of six, resulting in a significant cost reduction from the previous generation while supporting the signal frequencies of POWER9. POWER9 systems are the first to offer 16-Gb/s PCIe Gen4 and 25.8-Gb/s open coherent accelerator processor interface that interconnect the processor to the I/O, networking, and accelerators required for systems in the cognitive computing era. In this paper, we present the material and wiring technology needed to achieve the signal performance up to 25.8 Gb/s per channel, the package physical attributes, and the chip-package-system co-design methodology to achieve the increased signal density, minimize the crosstalk, and maximize the frequency while reusing the package form factors of the previous generation, IBM POWER8.

IBM POWER9 memory architectures for optimized systems

The IBM POWER9 processor chipset provides a variety of system memory architecture interfaces to enable highly differentiated system offerings: a high bandwidth, high capacity, highly reliable, buffered architecture; a compute-density-optimized direct DDR attach architecture; heterogeneous integration of graphics processing unit memory into the host system memory; and an agnostic, flexibly attached SCM architecture. In this paper, we explore these architectures and the targeted optimizations they provide for various classes of workloads. We also explore the development synergies and semiconductor physical design tradeoffs associated with the varying implementations, and finally, we describe several hypothetical systems that could be constructed by utilizing these memory architectures.

Improving IBM POWER8 Performance Through Symbiotic Job Scheduling

Symbiotic job scheduling, i.e., scheduling applications that co-run well together on a core, can have a considerable impact on the performance of processors with simultaneous multithreading (SMT) cores. SMT cores share most of their microarchitectural components among the co-running applications, which causes performance interference between them. Therefore, scheduling applications with complementary resource requirements on the same core can greatly improve the throughput of the system. This paper enhances symbiotic job scheduling for the IBM POWER8 processor. We leverage the existing cycle accounting mechanism to build an interference model that predicts symbiosis between applications. The proposed models achieve higher accuracy than previous models by predicting job symbiosis from throttled CPI stacks, i.e., CPI stacks of the applications when running in the same SMT mode to consider the statically partitioned resources, but without interference from other applications. The symbiotic scheduler uses these interference models to decide, at run-time, which applications should run on the same core or on separate cores. We prototype the symbiotic scheduler as a user-level scheduler in the Linux operating system and evaluate it on an IBM POWER8 server running multiprogram workloads. The symbiotic job scheduler significantly improves performance compared to both an agnostic random scheduler and the default Linux scheduler. Across all evaluated workloads in SMT4 mode, throughput improves by 12.4 and 5.1 percent on average over the random and Linux schedulers, respectively.

IBM Power9 Processor Architecture

The IBM Power9 processor has an enhanced core and chip architecture that provides superior thread performance and higher throughput. The core and chip architectures are optimized for emerging workloads to support the needs of next-generation computing. Multiple variants of silicon target the scale-out and scale-up markets. With a new core microarchitecture design, along with an innovative I/O fabric to support several accelerated computing requirements, the Power9 processor meets the diverse computing needs of the cognitive era and provides a platform for accelerated computing.

PATer: A Hardware Prefetching Automatic Tuner on IBM POWER8 Processor

Hardware prefetching on IBM's latest POWER8 processor is able to improve performance of many applications significantly, but it can also cause performance loss for others. The IBM POWER8 processor provides one of the most sophisticated hardware prefetching designs which supports 225 different configurations. Obviously, it is a big challenge to find the optimal or near-optimal hardware prefetching configuration for a specific application. We present a dynamic prefetching tuning scheme in this paper, named prefetch automatic tuner (PATer). PATer uses a prediction model based on machine learning to dynamically tune the prefetch configuration based on the values of hardware performance monitoring counters (PMCs). By developing a two-phase prefetching selection algorithm and a prediction accuracy optimization algorithm in this tool, we identify a set of selected key hardware prefetch configurations that matter mostly to performance as well as a set of PMCs that maximize the machine learning prediction accuracy. We show that PATer is able to accelerate the execution of diverse workloads up to 1.4×.

Transactional memory support in the IBM POWER8 processor

With multi-core processors, parallel programming has taken on greater importance. Traditional parallel programming techniques based on critical sections controlled by locking have several well-known drawbacks. To allow for more efficient parallel programming with higher performance, the IBM POWER8™ processor implements a hardware transactional memory facility. Transactional memory allows groups of load and store operations to execute and commit as a single atomic unit without the use of traditional locks, thereby improving performance and simplifying the parallel programming model. The POWER8 transactional memory facility provides a robust capability to execute transactions that can survive interrupts. It also allows non-speculative accesses within transactions, which facilitates debugging and thread-level speculation. Unique challenges caused by implementing transactional memory on top of the Power ISA (Instruction Set Architecture) weakly consistent memory model are addressed. We detail the Power ISA transactional memory architecture, the POWER8 implementation of this architecture, and two practical uses of this architecture—Transactional Lock Elision (TLE) and Thread-Level Speculation (TLS)—and provide performance results for these uses.

The cache and memory subsystems of the IBM POWER8 processor

In this paper, we describe the IBM POWER8™ cache, interconnect, memory, and input/output subsystems, collectively referred to as the “nest.” This paper focuses on the enhancements made to the nest to achieve balanced and scalable designs, ranging from small 12-core single-socket systems, up to large 16-processor-socket, 192-core enterprise rack servers. A key aspect of the design has been increasing the end-to-end data and coherence bandwidth of the system, now featuring more than twice the bandwidth of the POWER7® processor. The paper describes the new memory-buffer chip, called Centaur, providing up to 128 MB of eDRAM (embedded dynamic random-access memory) buffer cache per processor, along with an improved DRAM (dynamic random-access memory) scheduler with support for prefetch and write optimizations, providing industry-leading memory bandwidth combined with low memory latency. It also describes new coherence-transport enhancements and the transition to directly integrated PCIe® (PCI Express®) support, as well as additions to the cache subsystem to support higher levels of virtualization and scalability including snoop filtering and cache sharing.

IBM POWER8 performance features and evaluation

The IBM POWER8™ processor was designed for high performance on traditional server workloads as well as big data, analytics, and cloud workloads. In this paper, we describe key performance features of the IBM POWER8 processor. These include hardware assists that allow the POWER8 processor to automatically adapt to changing workloads by dynamically monitoring and tuning itself, enhancements to hardware instrumentation for performance monitoring, and performance improvements for encryption, virtualization, and I/O. We also describe the performance characteristics of a wide variety of applications, and we present the results of these applications running on POWER8 processor-based systems compared with previous generations of IBM Power Systems™.

Applications of the streamed storage format for sparse matrix operations

The streamed storage format for sparse matrices showed good performance improvement for sparse matrix and vector multiply (SpMV) compared with compressed sparse row (CSR) and block CSR (BCSR) formats, particularly on IBM Power processors. We extend the format to exploit single instruction multiple data (SIMD) instructions in order to utilize the vector unit, and discuss how the streamed formats perform on the Power7 processor, which is the first eight-core chip from IBM. The streamed format is then applied to two more operations of sparse matrices, successive over-relaxation (SOR) iteration sweeps and incomplete lower and upper (ILU) triangular solvers. Basic solvers are developed for them in the high-performance computing (HPC) package PETSc. Test results on the IBM Power7 processor show that the SIMD instructions improve the performance of the streamed storage format on SpMV. The format also accelerates SOR iteration sweeps and ILU matrix solvers, compared with the traditional BCSR format used in PETSc.

Concurrent Generation of Concurrent Programs for Post-Silicon Validation

The continuing trend toward increased parallelism in processor design can be seen in both the growing number of processor cores per system and in on-core hardware mechanisms that assist parallelism, such as multithreading and cache hierarchies. This complexity exacerbates the problem of ensuring the functional correctness of such hardware systems. The growing importance of post-silicon validation is leading to an emerging type of parallel application, namely, the hardware exerciser. We describe a method for exercising parallel hardware by generating pseudorandom concurrent test programs. The test generation is carried out on the tested parallel platform and thus the generator itself is also a concurrent program. We describe the challenges associated with this technology and the approach used by the Threadmill hardware exerciser, a tool developed for the post-silicon validation of the IBM POWER7 processor.

IBM POWER7 processor circuit design

The IBM POWER7® processor contains many innovative circuit ideas that enable advanced architectural features. A high-density embedded dynamic random access memory is used to provide 32 MB of level-3 cache. Improved input/output (I/O) links provide up to 50 GB/s of I/O bandwidth. An innovative phase-locked loop design allows for dynamic, per-core frequency variation, and unique multiport techniques for register files as well as six-transistor cell-based memory cells to support the superscalar multithreaded out-of-order processor.