Next-generation High-performance Computing Research Articles

Research on Multiplication Routine Based on Reconfigurable Four-Valued Logic Processor

Despite the indispensable role of traditional electronic computers in modern society, their limitations in parallel processing capabilities, bit-width constraints, and processor bit-width are becoming increasingly apparent, especially when handling large-scale datasets and complex computational tasks. Although hardware technology and algorithm optimization continue to advance, the arithmetic units of traditional computers—adders—remain constrained by carry delay and bit-width limitations. This bottleneck is particularly pronounced in multiplication operations, mainly when adders are used for partial product accumulation. However, since 2018, the emergence of a new type of Reconfigurable Four-Valued Logic Electronic Processor (RFLEP) has provided a potential solution to these traditional limitations. With its large processor bit-width, flexible bit grouping capabilities, and dynamic hardware function reconfiguration features, this processor has brought revolutionary changes to the field of computing. In this context, this paper proposes and implements a Reconfigurable Four-Valued Logic Multiplication Routine (RFLMR) tailored explicitly for the RFLEP. The RFLMR utilizes the Modified Signed-Digit (MSD) representation method in multi-valued logic combined with the M transformation in four-valued logic to generate partial products. These partial products are then efficiently summed in parallel using the JW-MSD parallel adder, achieving the rapid execution of multiplication operations. Experimental results demonstrate that the multiplication routine based on the RFLEP performs multiplication operations accurately and meets theoretical expectations regarding implementation efficiency and performance. This research not only provides new ideas for developing next-generation high-performance computing systems but also paves the way for exploring more efficient and powerful computing models, heralding a profound transformation in future computing technology.

Mathematical Analysis for GPU Framework for High Performance Computing with Improved Scalability, Reliability and Seamless Configurability

In the field of High Performance Computing (HPC), which is changing very quickly, using Graphics Processing Units (GPUs) has become essential for getting a lot of computing power and speed. This study gives a thorough mathematical analysis of a GPU system that is meant to improve scalability, stability, and configurability, all of which are very important for next-generation HPC apps. The suggested framework uses complex mathematical models and methods to make the best use of GPU resources, cut down on delay, and guarantee strong performance across a wide range of tasks. Scalable parallel processing methods are built into the framework so it can adapt to changing computing needs, making the most of speed and resource use. Fault-tolerant methods that lessen the effects of hardware breakdowns and computing mistakes also improve reliability, making sure that results are always correct and consistent. The framework can be set up in different ways because it is based on flexible design, which makes it easy to change and adapt to the needs of each application without affecting speed. In diverse computer settings, where different apps may have different working needs, this ability to change is very important. The mathematical analysis includes performance measures like speed, delay, and mistake rates, which give a solid basis for judging how well the system works. Additionally, tests comparing the suggested system to current GPU tools show that it is more reliable and scalable. This study helps make better, more reliable computer systems by looking at some of the biggest problems in GPU-based high-performance computing. The results show that using this method can greatly improve the skills of HPC systems, which can lead to progress in scientific study, data analysis, and complex models. In the end, this work shows how advanced GPU systems can help drive innovation in HPC, leading to more powerful, reliable, and flexible computing options.

Codesign for Extreme Heterogeneity: Integrating Custom Hardware With Commodity Computing Technology to Support Next-Generation HPC Converged Workloads

The future of high-performance computing (HPC) will be driven by the convergence of physical simulation, artificial intelligence, machine learning, and data science computing capabilities. While computational performance gains afforded by technology scaling, as predicted by Moore's Law, have enabled large-scale HPC system design and deployment using commodity CPU and GPU processing components, emerging technologies will be required to effectively support such converged workloads. These emerging technologies will integrate commodity computing components with custom processing and networking accelerators into ever-more heterogeneous architectures resulting in a diverse ecosystem of industry technology developers, university, and U.S. Government researchers. In this article, we describe efforts at the U.S. Department of Energy's Pacific Northwest National Laboratory to construct an end-to-end codesign framework that lays a groundwork for such an ecosystem, including notable outcomes, remaining challenges, and future opportunities.

A Comprehensive Methodology to Optimize FPGA Designs via the Roofline Model

With reconfigurable fabrics delivering increasing performance over the years, Field-Programmable Gate Arrays (FPGAs) are becoming an appealing solution for next-generation High-Performance Computing (HPC) systems. However, in order to gain traction among traditional von Neumann architectures, the optimization process of Field-Programmable Gate Array (FPGA) designs should be further abstracted to a higher level. In fact, while High-Level Synthesis (HLS) already provides a handy way to write FPGA code with common high-level languages, substantial effort and expertise are still required to optimize the resulting FPGA design for the underlying hardware. To overcome this problem, we propose a semi-automated performance optimization methodology based on a Hierarchical Roofline model for FPGAs. System-wide and applications-specific optimizations such as off-chip memory transfer and data locality optimizations are guided by the FPGA Roofline model whereas FPGA-specific optimizations are automatically searched by a Design Space Exploration (DSE) engine. We demonstrate the way this methodology allows to easily analyze and optimize to peak system performance a wide set of applications ranging from particle methods, wavefront algorithms, and sparse arithmetic computations. In addition, we prove that the integrated Design Space Exploration (DSE) engine achieves a 14.36x maximum speedup if compared to previous automated solutions in the literature.

Comparative evaluation of deep learning workloads for leadership-class systems

Deep learning (DL) workloads and their performance at scale are becoming important factors to consider as we design, develop and deploy next-generation high-performance computing systems. Since DL applications rely heavily on DL frameworks and underlying compute (CPU/GPU) stacks, it is essential to gain a holistic understanding from compute kernels, models, and frameworks of popular DL stacks, and to assess their impact on science-driven, mission-critical applications. At Oak Ridge Leadership Computing Facility (OLCF), we employ a set of micro and macro DL benchmarks established through the Collaboration of Oak Ridge, Argonne, and Livermore (CORAL) to evaluate the AI readiness of our next-generation supercomputers. In this paper, we present our early observations and performance benchmark comparisons between the Nvidia V100 based Summit system with its CUDA stack and an AMD MI100 based testbed system with its ROCm stack. We take a layered perspective on DL benchmarking and point to opportunities for future optimizations in the technologies that we consider.

Performance and energy consumption of HPC workloads on a cluster based on Arm ThunderX2 CPU

In this paper, we analyze the performance and energy consumption of an Arm-based high-performance computing (HPC) system developed within the European project Mont-Blanc 3. This system, called Dibona, has been integrated by ATOS/Bull, and it is powered by the latest Marvell’s CPU, ThunderX2. This CPU is the same one that powers the Astra supercomputer, the first Arm-based supercomputer entering the Top500 in November 2018. We study from micro-benchmarks up to large production codes. We include an interdisciplinary evaluation of three scientific applications (a finite-element fluid dynamics code, a smoothed particle hydrodynamics code, and a lattice Boltzmann code) and the Graph 500 benchmark, focusing on parallel and energy efficiency as well as studying their scalability up to thousands of Armv8 cores. For comparison, we run the same tests on state-of-the-art x86 nodes included in Dibona and the Tier-0 supercomputer MareNostrum4. Our experiments show that the ThunderX2 has a 25% lower performance on average, mainly due to its small vector unit yet somewhat compensated by its 30% wider links between the CPU and the main memory. We found that the software ecosystem of the Armv8 architecture is comparable to the one available for Intel. Our results also show that ThunderX2 delivers similar or better energy-to-solution and scalability, proving that Arm-based chips are legitimate contenders in the market of next-generation HPC systems.

Advanced Low-K Polymer Dielectric Materials and Interfaces for Fine-Pitch Re-Distribution-Layer (RDL) to Enable 2.5D and Fan-out Packages

The rapid increase of devices with artificial intelligence (AI) and machine learning capabilities is driving the demand for growth in compute performance. With transistor scaling reaching physical limitations, the semiconductor industry is looking to system-scaling to deliver the performance improvements. Advanced packaging architectures capable of supporting these applications are 2.5D interposers, 3D IC stacking, chiplets and fan-out based packaging. All of these architectures require high-density interconnects capable of routing >200 IOs/mm/layer. This talk demonstrates advanced ultra-thin ( <10 µm), low-Dk (< 3.0) polymer dielectric material candidates capable of achieving I/O densities of 500 IOs/mm/layer. Current commercially available solutions based on silicon interposers suffer from conductor and dielectric losses because of restrictions on the dielectric material (SiO2). Polymer based dielectrics offer a compelling alternative because of the potential to reduce the capacitance of the traces and improve power efficiency alongside compatibility with low-cost panel-scalable processes. This idea is explored will be explored in this talk and we will demonstrate through modelling and characterization that low-Dk polymer dielectrics are a superior candidate to current dielectric material candidates The choice of polymer dielectric is driven by technology trends towards a)high-density fine-pitch routing b)system level thermo-mechanical reliability c)panel-scalable processing. This work examines four dielectric material candidates comprising of 3 material classes, namely photo-sensitive epoxy, non-photo-sensitive epoxy and BCB for next-generation interconnect technology. High-speed electrical simulations indicate that low-Dk materials can support data rates of >10Gbps which overcomes the limitations faced by conventional high Dk epoxy dielectrics and SiO2 alternatives. The biggest challenge in moving towards low-Dk materials is that these polymers are typically non-polar and do not adhere well which poses challenges in package integration. This work proposes a unique approach to improve polymer/metal interfacial adhesion by creating a hybrid layer with tailorable material properties using vapor-phase-infiltration (VPI). VPI exposes the polymeric material to a metal-organic reactant such as trimethyl aluminium (TMA) in a heated environment. Diffusional kinetics of the reactant are highly dependent on each step of the sequential process such as temperature, time of hold, exposure time, and co-reactants used. This process is superior compared to Atomic Layer deposition (ALD) in promoting the diffusion of organometallic precursors into the polymer subsurface because of a hold-period after exposure. VPI thereby provides a highly adaptable process to create tunable hybrid interfaces with target properties. VPI-treated polymers showed a 3X improvement in adhesion strength at the polymer/metal interface. This work dives into the kinetic factors required to achieve this adhesion improvement and characterizes the corresponding material structure changes. In summary, advanced low-Dk polymer dielectrics capable of meeting next-generation high-performance computing requirements are discussed. This work evaluates the key performance benefits from moving to low-Dk dielectric materials and tackles a critical challenge of integrating these materials as re-distribution layer (RDL) dielectrics by demonstrating 3X improvement in adhesion using VPI.

Low polarization-dependent-loss double-layer grating coupler for three-dimensional photonic integration

Dense integration of silicon photonic devices is the key enabling technology for next-generation high-performance computers. In this paper, we propose a double-layer grating coupler for the future multi-layer photonic integrated system. With a pair of one-dimensional (1-D) gratings orthogonally vertical-stacked, the device can couple light between single-mode fiber and two separate waveguide layers, realizing polarization splitting and rotation in different layers simultaneously. Benefiting from the perfectly vertical coupling scheme and optimal fiber position (OFP) overlapping design, the coupling efficiency peak of the upper and lower grating are calculated to be 28% and 25% respectively. Assisted with a DBR bottom mirror, the coupling efficiency of the two gratings can be further enhanced to 33.2% and 28.9% respectively. For real application, the misalignment tolerance on grating overlapping and the incidence angle are carefully investigated. To achieve power balance between the two grating layers, the impact of top cladding thickness is analyzed and an optimized thickness of 0.6μm is chosen. From the simulation result of a back-to-back DLGC circuit, the coupling performance shows considerable robustness to the input polarization state and the PDL is under 0.5 dB within the wavelength range from 1536 nm to 1558 nm. Such a device can be a very promising solution to achieve polarization-independent coupling in the future three-dimensional (3-D) high density photonic integrated circuits.

Task Scheduling Techniques for Asymmetric Multi-Core Systems

As performance and energy efficiency have become the main challenges for next-generation high-performance computing, asymmetric multi-core architectures can provide solutions to tackle these issues. Parallel programming models need to be able to suit the needs of such systems and keep on increasing the application’s portability and efficiency. This paper proposes two task scheduling approaches that target asymmetric systems. These dynamic scheduling policies reduce total execution time either by detecting the longest or the critical path of the dynamic task dependency graph of the application, or by finding the earliest executor of a task. They use dynamic scheduling and information discoverable during execution, fact that makes them implementable and functional without the need of off-line profiling. In our evaluation we compare these scheduling approaches with two existing state-of the art heterogeneous schedulers and we track their improvement over a FIFO baseline scheduler. We show that the heterogeneous schedulers improve the baseline by up to 1.45 $\times$ in a real 8-core asymmetric system and up to 2.1 $\times$ in a simulated 32-core asymmetric chip.

Modeling and Simulation of Extreme-Scale Fat-Tree Networks for HPC Systems and Data Centers

As parallel and distributed systems are evolving toward extreme scale, for example, high-performance computing systems involve millions of cores and billion-way parallelism, and high-capacity storage systems require efficient access to petabyte or exabyte of data, many new challenges are posed on designing and deploying next-generation interconnection communication networks in these systems. Fat-tree networks have been widely used in both data centers and high-performance computing (HPC) systems in the past decades and are promising candidates of the next-generation extreme-scale networks. In this article, we present FatTreeSim, a simulation framework that supports modeling and simulation of extreme-scale fat-tree networks with the goal of understanding the design constraints of next-generation HPC and distributed systems and aiding the design and performance optimization of the applications running on these systems. We have systematically experimented FatTreeSim on Emulab and Blue Gene/Q and analyzed the scalability and fidelity of FatTreeSim with various network configurations. On the Blue Gene/Q Mira, FatTreeSim can achieve a peak performance of 305 million events per second using 16,384 cores. Finally, we have applied FatTreeSim to simulate several large-scale Hadoop YARN applications to demonstrate its usability.

SIERRA—Simulation environment for memory redundancy algorithms

Extreme-scale computer systems take advantage of large arrays of general-purpose multicore processors coupled with specialized manycore accelerators. In order to support complex applications and correctly feed such processing elements, increasingly larger memory cores are integrated at different levels of the hierarchy. However, the adoption of increasingly aggressive manufacturing processes makes the memory sub-system particularly sensitive to faults. Error correcting codes (ECCs) allow the memory to recover from faults at run-time without interfering with the application execution. However, due to the loss of performance introduced every time an error must be corrected, the persistence of faults requires a more radical repair approach in which faulty cells are physically replaced by spare ones. Memory redundancy analysis (MRA) algorithms are used to drive the allocation process of spare resources. Many one-dimensional and two-dimensional MRAs have been proposed, but tools for evaluating their recovering capability are still not well established. This paper presents SIERRA, a simulation environment for precisely evaluating the repair efficiency of an MRA considering different fault signatures and faulty memory configurations. Our simulation engine provides a precise estimation of the MRA quality by analyzing the behavior of the MRA on several faulty memory configurations. To this end, different parameters such as the area of the memory blocks and the defect density are taken into account. The evaluation of the quality of an MRA takes into account its repairing capability, the power consumption derived from its execution, and the area overhead. Thanks to the use of a database for storing information, our tool is able to speed-up the simulation process by distributing it among several nodes. All these features make SIERRA essential in supporting the design of next-generation high-performance computers.

Exploring the Design Tradeoffs for Extreme-Scale High-Performance Computing System Software

Owing to the extreme parallelism and the high component failure rates of tomorrow's exascale, high-performance computing (HPC) system software will need to be scalable, failure-resistant, and adaptive for sustained system operation and full system utilizations. Many of the existing HPC system software are still designed around a centralized server paradigm and hence are susceptible to scaling issues and single points of failure. In this article, we explore the design tradeoffs for scalable system software at extreme scales. We propose a general system software taxonomy by deconstructing common HPC system software into their basic components. The taxonomy helps us reason about system software as follows: (1) it gives us a systematic way to architect scalable system software by decomposing them into their basic components; (2) it allows us to categorize system software based on the features of these components, and finally (3) it suggests the configuration space to consider for design evaluation via simulations or real implementations. Further, we evaluate different design choices of a representative system software, i.e. key-value store, through simulations up to millions of nodes. Finally, we show evaluation results of two distributed system software, Slurm++ (a distributed HPC resource manager) and MATRIX (a distributed task execution framework), both developed based on insights from this work. We envision that the results in this article help to lay the foundations of developing next-generation HPC system software for extreme scales.

On noise and the performance benefit of nonblocking collectives

Relaxed synchronization offers the potential for maintaining application scalability, by allowing many processes to make independent progress when some processes suffer delays. Yet the benefits of this approach for important parallel workloads have not been investigated in detail. In this paper, we use a validated simulation approach to explore the noise-mitigation effects of idealized nonblocking collectives, in workloads where these collectives are a major contributor to total execution time. Although nonblocking collectives are unlikely to provide significant noise mitigation to applications in the low operating system noise environments expected in next-generation high-performance computing systems, we show that they can potentially improve application runtime with respect to other noise types.

3D Stacked Cache Data Management for Energy Minimization of 3D Chip Multiprocessor

In this model a runtime cache data mapping is discussed for 3-D stacked L2 caches to minimize the overall energy of 3-D chip multiprocessors (CMPs). The suggested method considers both temperature distribution and memory traffic of 3-D CMPs. Experimental result shows energy reduction achieving up to 22.88% compared to an existing solution which considers only the temperature distribution. New tendencies envisage 3D Multi-Processor System-On-Chip (MPSoC) design as a promising solution to keep increasing the performance of the next-generation high performance computing (HPC) systems. However, as the power density of HPC systems increases with the arrival of 3D MPSoCs with energy reduction achieving up to 19.55% by supplying electrical power to the computing equipment and constantly removing the generated heat is rapidly becoming the dominant cost in any HPC facility.

Silicon Photonic Transceiver Circuits With Microring Resonator Bias-Based Wavelength Stabilization in 65 nm CMOS

Photonic interconnects are a promising technology to meet the bandwidth demands of next-generation high-performance computing systems. This paper presents silicon photonic transceiver circuits for a microring resonator-based optical interconnect architecture in a 1 V standard 65 nm CMOS technology. The transmitter circuits incorporate high-swing ( 2V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pp</sub> and 4V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pp</sub> ) drivers with nonlinear pre-emphasis and automatic bias-based tuning for resonance wavelength stabilization. An optical forwarded-clock adaptive inverter-based transimpedance amplifier (TIA) receiver trades off power for varying link budgets by employing an on-die eye monitor and scaling the TIA supply for the required sensitivity. At 5 Gb/s operation, the 4V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pp</sub> transmitter achieves 12.7 dB extinction ratio with 4.04 mW power consumption, excluding laser power, when driving wire-bonded modulators designed in a 130 nm SOI process, while a 0.28 nm tuning range is obtained at 6.8 μW/GHz efficiency with the bias-based tuning scheme implemented with the 2V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pp</sub> transmitter. When tested with a wire-bonded 150 fF p-i-n photodetector, the receiver achieves -9 dBm sensitivity at a BER=10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-9</sup> and consumes 2.2 mW at 8 Gb/s. Testing with an on-die test structure emulating a low-capacitance waveguide photodetector yields 17 μA <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pp</sub> sensitivity at 10 Gb/s and more than 40% power reduction with higher input current levels.

Ring Resonator Modulators in Silicon for Interchip Photonic Links

Silicon photonics is the most promising pathway to achieve >10 Tb/s off-chip I/O bandwidth required by next-generation high-performance computing and switching systems. Ring resonator modulators offer the advantages of small footprint, low power, high efficiency, low loss, high speed, and CMOS compatibility for silicon photonic links. This paper presents an in-depth discussion of practical microring modulators in silicon, covering their performance metrics, design tradeoffs, optimization, p-n junction geometries, complex ring configurations, and tuning solutions. Various demonstrated Si ring modulators are reviewed and potential future developments are briefly discussed.

Simulation of cardiac electrophysiology on next-generation high-performance computers

Models of cardiac electrophysiology consist of a system of partial differential equations (PDEs) coupled with a system of ordinary differential equations representing cell membrane dynamics. Current software to solve such models does not provide the required computational speed for practical applications. One reason for this is that little use is made of recent developments in adaptive numerical algorithms for solving systems of PDEs. Studies have suggested that a speedup of up to two orders of magnitude is possible by using adaptive methods. The challenge lies in the efficient implementation of adaptive algorithms on massively parallel computers. The finite-element (FE) method is often used in heart simulators as it can encapsulate the complex geometry and small-scale details of the human heart. An alternative is the spectral element (SE) method, a high-order technique that provides the flexibility and accuracy of FE, but with a reduced number of degrees of freedom. The feasibility of implementing a parallel SE algorithm based on fully unstructured all-hexahedra meshes is discussed. A major computational task is solution of the large algebraic system resulting from FE or SE discretization. Choice of linear solver and preconditioner has a substantial effect on efficiency. A fully parallel implementation based on dynamic partitioning that accounts for load balance, communication and data movement costs is required. Each of these methods must be implemented on next-generation supercomputers in order to realize the necessary speedup. The problems that this may cause, and some of the techniques that are beginning to be developed to overcome these issues, are described.

LSI-to-LSI Signal Transmission at 10 Gb/s/ch Using Ultra-High Density Optoelectronic Modules

As the signal speed of LSIs increases, so does the need to replace copper interconnects with optical interconnects, especially for high performance computing (HPC) . The LSIs of next-generation HPC have as many as approximately 1, 000 I/O signals, so the optoelectronic module for them has to be small. We developed high-density optoelectronic modules that have 12 channels operating at 10Gbps with a 4.5×4.5-mm substrate that has small electrode pads for mounting optical devices on the side face. We demonstrated 10-Gbps transmission between two prototype LSIs using the optoelectronic modules. 10-Gbps signal was transmitted from an LSI to another LSI through electrical-optical-electrical signal path, and error free operation (BER<1E-12) was achieved.