IBM Z14 Research Articles

IBM z15: Physical design improvements to significantly increase content in the same technology

The IBM Z processor continues to improve over previous System Z processors, but for the first time it does so without a technology improvement as the baseline enabler. The IBM z15 was designed in the same 14-nm High-Performance GLOBALFOUNDRIES technology as the IBM z14 and yet still added 20% more cores, doubled the L3 cache, and increased the L2 cache by a third while also adding a third peripheral component interconnect express (PCIe) port to the chip and an elliptic curve cryptography engine into each core. This article discusses the design, tool, and methodology enhancements required to increase the design content so significantly while maintaining the chip size and power limits from the previous z14 design. This article also discusses other design and methodology improvements that were made possible via the deeper understanding of the technology and how to more fully leverage it in a second generation.

IBM z15: Improved data center density and energy efficiency, new system packaging, and modeling

The IBM z15 is designed to meet the requirements of a range of data centers, while reducing costs through increased density, configuration flexibility, and cooling efficiency. The z15 is a continuation and broadening of the physical transformation of the mainframe that began with the IBM z14 ZR1/LR1, which introduced the new “true 19-in” frame. A maximum configuration z15 delivers greater than 30% additional compute capacity per watt than z14, and maintains approximately the same maximum system footprint, while enabling significant floor space reduction for most configurations. The z15 introduces the choice of integrated 2N power using either intelligent power distribution units or bulk power, also supporting most data centers including hot/cold-aisle containment, raised-floor and nonraised-floor, and top and bottom-exit I/O and power. The z15 supports the ASHRAE A3 (fourth edition) environment, providing efficiency advantages by reducing humidification requirements. The z15 maintains the value of a system that is preconfigured/pretested before shipping. Innovations in packaging, I/O cabling, controls, and testing are put in the context of the latest data center trends. The capabilities of new tools to estimate power, weight, airflow, heat extracted to water for water-cooled systems as well as 3-D and computational fluid dynamics models to aid in the planning for the system are described.

Design of the IBM z15 microprocessor

The latest-generation IBM Z processor provides enhanced performance and compute capacity compared to its IBM z14 predecessor. This article describes some of the major improvements in both process and design including out-of-order load-and-store sequencing, single-instruction multiple-data and floating point enhancements, a new modulo arithmetic engine for accelerating elliptic curve cryptography, a hardware sort accelerator, and a workflow that modernized the development of these features. Outside of the central processing unit (CPU), the cache sizes have increased on all levels, and each processor chip now contains 12 CPUs. System topology changes have been introduced allowing up to five drawers to exist in a fully populated system. The processor cache subsystem includes numerous improvements in the area of fetch, store, and cache management policies aimed at speeding up both traditional data serving workloads and highly virtualized environments alike.

IBM z14: Processor Characterization and Power Management for High-Reliability Mainframe Systems

The IBM z14 is the latest update in the storied history of IBM mainframes. Reliability, availability, security, and scalability are the foundation of the IBM mainframe line. System reliability and availability targets are in excess of 10 years, requiring rigorous chip characterization processes. In this paper, we discuss some of the many processes used to ensure that lifetime. An additional part of this reliability is power management (PM). The 5.2-GHz high-power design of the central processor chip requires advanced on-die PM capabilities to adapt to power intensive instruction streams. We also discuss a number of improvements to the critical path monitoring design used to manage power supply voltage droops, reducing response time and the impact on system performance. Finally, we compare a set of simulations and hardware results to validate our power fluctuation models.

IBM z14: Improved datacenter characteristics, energy efficiency, and packaging innovation

The IBM z14 mainframe provides significant improvements in datacenter physical characteristics, flexibility, robustness, energy efficiency, and modeling capability relative to its predecessor, the IBM z13. In this paper, we consider these improvements in the context of the latest trends in the commercial datacenter and describe the innovations in packaging, controls, testing, modeling, and tools that enabled them. The z14 supports the American Society of Heating Refrigeration and Air Conditioning Engineers class A3 maximum air-inlet temperature of 40 °C, versus 35 °C for the IBM z13. Water-cooled models have increased maximum inlet water temperature. A maximum-configuration IBM z14 delivers 23% or more traditional compute capacity per watt than the z13. The z14 offers a cover set option that has a total system depth of 1.486 m versus 1.87 m for z13, which is much closer to the 1.20-m depth of many datacenter racks. The z14 has a high level of physical robustness with respect to shipping and operating shock and vibration, as well as earthquakes. We discuss how this robustness was achieved through design, modeling, and testing. We also review the capabilities of tooling to accurately estimate system power, weight, and airflow, as well as the heat extracted to water in the water-cooled systems.

Large-scale application of IBM Design Thinking and Agile development for IBM z14

In this paper, we describe the application of IBM Design Thinking and Agile development for the IBM z14 project. We discuss the adoption of IBM Design Thinking practices and its impact on the development and client reception of IBM z14. Furthermore, we show how the design direction of the IBM z14 system was iteratively shaped by involving key members of client organizations (“sponsor users”) early in the development process. Next, we discuss how applying Agile practices in a hardware/firmware environment—with iterative development and test, playbacks, and continuous integration—allowed for faster course correction and delivery on a challenging schedule. This paper provides a case study for the adoption of IBM Design Thinking and Agile practices, which were originally designed for small team operations, and the challenge of applying them to a large-scale development organization such as the IBM team developing the z14 with more than 1,000 engineers.

IBM z14 and the Cognitive Era

The IBM z14 is designed for cognitive and analytics processing to support business use cases. With the explosive growth in the amount and richness of data and data-driven business processes, being able to process, analyze, and apply machine learning in a timely and efficient manner is often a business necessity. Improvements to the single-instruction–multiple-data (SIMD) facility, as well as improvements to Java garbage collection, provide improved support for different software packages. Software such as Apache Spark has been enabled for both z/OS and Linux on z to provide the software infrastructure required for cognitive and analytics processing, while new software such as IBM Machine Learning for z/OS provides additional assistance for data scientists. End-to-end cross-platform solutions that require data that is stored on the z14 can also be improved using these new software and hardware capabilities. This paper explores the various hardware capabilities of z14, as well as the enabling software and end-to-end solutions.

IBM z14: Enabling physical design in 14-nm technology for high-performance, high-reliability microprocessors

The IBM z14 design was built with the 14-nm high-performance silicon-on-insulator (SOI) technology of GLOBALFOUNDRIES. This was the first technology node after IBM transitioned from its integrated fabrication facility to operating in a fabless environment, driving significant changes to design processes and methodology. In addition to this partnership, the 14-nm technology introduced significant changes relative to previous technology nodes, including the introduction of fin-shaped field-effect transistors, the use of double patterning for the lowest back-end-of-line layers, and the introduction of middle-of-line layers to exploit contact layers for local interconnects. This combination of technical and business challenges required numerous large-scale innovations for our design, design team, and design methodologies. In this paper, we provide a survey of these innovations, including the fin-based standard cell image, deeply scaled SOI self-heating/electromigration verification, routing strategies to handle double-patterning with interlayer via awareness, fill automation to enable simultaneous design of multiple layers of hierarchy, and high-performance array design with the voltage and noise limitations of the 14-nm technology node.

IBM z14: Advancing the I/O storage and networking channel adapter

The IBM z14 system introduces a new input/output adapter that supports two storage protocols as well as a networking protocol. This is the first time three protocols have been supported on one application-specific integrated circuit (ASIC). The three protocols supported are Fibre Connection, Fibre Channel Protocol, and Ethernet. In addition to consolidation, enhancements included substantial performance improvements, reductions in power, and updates to both the technology and multiple intellectual property interface standards. Due to the increased complexity of this ASIC, new verification methodologies were introduced that also incorporated firmware running on programmable sequence engines. This enabled testing earlier in the development cycle, referred to as a shift-left transformation, so our firmware and software development teams could start coding even before the actual hardware was built.

An optimized blockchain solution for the IBM z14

Blockchain is a technology that has been available for several years and has already had several generations. Multiple implementations exist. In this paper, we describe how one of them, the Hyperledger Fabric, is used today to demonstrate the wide usage of the technology for multiple markets beyond finance and cryptocurrency. The use cases highlight the relevance of security—due to the type of data and the distributed foundation of blockchain—and of system performance to enable the use cases in production. Security and performance are the base for the architecture of blockchain on the IBM LinuxONE systems. The Secure Service Container prevents tampering, and the cryptocard usage hardens the solution—and the performance optimization in the compiler, cryptoinstructions, and large caches provides the performance. These optimizations have been further extended with the IBM z14 beyond previous IBM Z generations. In this paper, we describe how a cluster topology is used to implement a high-availability service that scales dynamically from small networks to large ones using the hot plug capabilities of the IBM LinuxONE systems.

Preface: IBM z14 Design and Technology

This issue describes the innovative design and technology of the IBM z14, the latest IBM mainframe, with its significant new capabilities, including pervasive encryption, analytics and machine learning, platform simplification, enhanced system capacity and performance, significant reduction in input/output (I/O) latency, data serving, reliability, robustness, and new semiconductor process technology. The topics covered in this issue also consider client value, solutions, and business motivation, along with additional new technologies and innovations across the stack.

IBM z14 design methodology enhancements in the 14-nm technology node

In designing the IBM z14 microprocessor chipset, we discarded many of our previous assumptions and processes in favor of newer, more radical approaches. These new approaches were the result of learning from previous designs as well as this design's performance and schedule requirements. In this paper, we discuss some of the more significant changes to our methodology, including a dramatic departure from our very hierarchical integration design style to a flatter, more nimble approach. We also discuss improvements to our design's power management architecture for managing power-supply noise. Finally, we discuss changes to our simulation environment, targeted at increasing both the number of simulation cycles and simulation logic coverage.

Performance innovations in the IBM z14 platform

The IBM z14 platform brings together numerous innovations that collectively provide considerable performance gains over prior designs for the full spectrum of modern workloads being run on today's enterprise systems. These workloads range from traditional online transaction processing and batch workloads to Linux-based and cloud-based workloads, leveraging advances in analytics, mobile transaction processing, social data mining, cognitive applications, security (blockchain), and other areas. The performance gains in the z14 platform required advancements in the processor hardware microarchitecture, physical design, and instruction-set architecture, as well as in the firmware and hypervisor stacks. The result is a comprehensive platform solution that enables efficient multiprocessor scaling for large single-image partitions and mixed-workload multipartition environments within a single platform, while remaining responsive to the “bursty” demands of batch and transaction-based workloads. The performance gains present in the z14 are a direct result of the collaborative efforts between diverse teams within IBM working together to produce advancements in hardware, firmware, and software that outpace the gains of the individual contributions. In this paper, we discuss these results with a focus on the client value delivered through system performance and workload responsiveness.

ZHyperLink: Low-latency I/O for Db2 on IBM Z and DS8880 storage

zHyperLink is a new low-latency I/O capability for storage area networks (SANs). When using zHyperLink, the IBM z14 processor, along with the DS8880 Storage System (IBM's latest enterprise-class storage for mainframe attachment), delivers up to a 90% reduction in storage I/O service times compared with fiber channel transport technology. zHyperLink I/O service times eliminate the need for an operating system task switch for most read I/O operations, fundamentally improving application response times and changing system scaling behavior. Initial use of zHyperLink on IBM Z, IBM DS8880 storage, and z/OS accelerates the Db2 for z/OS relational database transaction processing information management system hierarchal database and reduces the elapsed time for Virtual Storage Access Method applications. These improvements occur without any application changes. In this paper, we explain the strategic importance of a low-latency I/O capability for the SAN and describe the z14 architecture, hardware, firmware, and software implementation concepts and the preliminary performance results for Db2.

IBM z14 Parallel Sysplex and coupling technology

IBM z14 Parallel Sysplex and coupling technology

Platform simplification for IBM z14

In this paper, we describe our continued focus on simplifying the IBM Z environment and the key changes made across the mainframe stack, including platform management, hypervisors, and operating systems. For the IBM z14 program, the first goal was to enable system administrators, with no mainframe experience, to provision resources and deploy Linux workloads on the mainframe in minutes using their current skills. Further goals include providing a mobile experience for users to securely monitor and manage their systems from “anywhere,” addressing critical user pain points in the configuration of server time protocol using a topological visualization, securing the hardware management console (HMC) with two-factor authentication, and modernizing the main user workspace. Finally, the concept of attaching storage resources to an IBM Z environment has been changed to provide a consistent end-to-end experience for both the novice system administrator and the storage administrator. Additionally, the two IBM Design hallmark projects, IBM Dynamic Partition Manager and IBM zHMC.Next, are also critical to achieve these goals. Finally, applying IBM Design Thinking and Agile practices, and establishing radical collaboration with our customers, is essential to deliver these meaningful outcomes.

Design of the IBM z14 microprocessor

The latest-generation IBM Z processor provides enhanced performance and compute capacity compared to its IBM z13 predecessor. This paper describes some of the major improvements that include an additional perceptron branch predictor, a completely redesigned translation engine that is tightly integrated into the core pipeline, and an integrated level-1 cache directory and translation lookaside buffer design. Outside of the central processing unit (CPU), the cache sizes have increased on each cache level, and each processor chip now contains 10 CPUs. The system topology has been optimized to improve cache transfer latencies for workloads spanning multiple processor chips. The bus interfaces between the chips have been redesigned to improve peak bus traffic handling. In combination, these enhancements provide significant performance improvements in traditional data-serving workloads, as well as in virtualized Linux environments running database, analytic, and cognitive workloads.

New database compression assists in the IBM z14 processor

In this paper, we describe how the IBM z14 processor, together with Db2 for z/OS Version 12, can improve data compression rates and thus reduce data storage requirements and cost for large databases. The new processor improves on the compression hardware accelerator available in earlier IBM Z generations by adding new hardware algorithms that increase the compression ratio and extend the applicability to additional data structures in Db2 for z/OS databases. A new entropy coding step employed after Ziv–Lempel compression reduces the size of data compressed with the prior algorithms by 30% on average. Moreover, the new order-preserving compression algorithm enables database index compression, reducing index sizes by roughly 30%. This results in an overall improvement of 30% of the database size for many applications, with associated benefits in storage requirements, input/output bandwidth, and buffer pool efficiency.