Hybrid CPU-GPU Research Articles

Mixed-precision pre-pivoting strategy for the LU factorization

This paper investigates the efficient application of half-precision floating-point (FP16) arithmetic on GPUs for boosting LU decompositions in double (FP64) precision. Addressing the motivation to enhance computational efficiency, we introduce two novel algorithms: Pre-Pivoted LU (PRP) and Mixed-precision Panel Factorization (MPF). Deployed in both hybrid CPU-GPU setups and native GPU-only configurations, PRP identifies pivot lists through LU decomposition computed in reduced precision and subsequently reorders matrix rows in FP64 precision before executing LU decomposition without pivoting. Two variants of PRP, namely hPRP and xPRP, are introduced, differing in their computation of pivot lists in full half-precision or mixed half-single precision. The MPF algorithm generates FP64 LU factorization while internally utilizing hPRP for panel factorization, showcasing accuracy on par with standard DGETRF but with superior speed. The study further explores auxiliary functions required for the native mode implementation of PRP variants and MPF.

High-Performance Flow Classification of Big Data Using Hybrid CPU-GPU Clusters of Cloud Environments

High-Performance Flow Classification of Big Data Using Hybrid CPU-GPU Clusters of Cloud Environments

Two-dimensional quantum lattice models via mode optimized hybrid CPU-GPU density matrix renormalization group method

Two-dimensional quantum lattice models via mode optimized hybrid CPU-GPU density matrix renormalization group method

Direct reduction of iron-ore with hydrogen in fluidized beds: A coarse-grained CFD-DEM-IBM study

Hydrogen metallurgy technology uses hydrogen as the reducing agent instead of carbon reduction, which is one of the important ways to reduce carbon dioxide emissions and ensure the green and sustainable development of iron and steel industry. Due to the advantages of high gas-solid contact efficiency and outstanding mass and heat transfer, direct reduction of iron ore in fluidized beds has attracted much attention. Based on the three-layer unreacted shrinking core model and considering the diversity of particle structure during the reaction process, a multi-reaction path (multi-resistance network) model was established. In this study, a coarse-grained CFD-DEM-IBM solver based on hybrid CPU–GPU computing is developed to simulate the direct reduction process of two kinds of iron ore with hydrogen in fluidized beds, where the developed unreacted shrinking core model is used to model the reduction reactions, a coarse-grained model and multiple GPUs enable the significant acceleration of particle computation, and the immersed boundary method (IBM) enables the use of simple mesh even in complex geometries of reactors. The predicted results of particle reduction degree are in good agreement with the experimental values, which proves the correctness of the CFD-DEM-IBM solver. In addition, the effects of reaction kinetic parameters and operating temperature on particle reduction degree are also investigated. Present study provides a method for digital design, optimization and scale-up of ironmaking reactors.

A comparison of Algebraic Multigrid Bidomain solvers on hybrid CPU–GPU architectures

The numerical simulation of cardiac electrophysiology is a highly challenging problem in scientific computing. The Bidomain system is the most complete mathematical model of cardiac bioelectrical activity. It consists of an elliptic and a parabolic partial differential equation (PDE), of reaction–diffusion type, describing the spread of electrical excitation in the cardiac tissue. The two PDEs are coupled with a stiff system of ordinary differential equations (ODEs), representing ionic currents through the cardiac membrane. Developing efficient and scalable preconditioners for the linear systems arising from the discretization of such computationally challenging model is crucial in order to reduce the computational costs required by the numerical simulations of cardiac electrophysiology. In this work, focusing on the Bidomain system as a model problem, we have benchmarked two popular implementations of the Algebraic Multigrid (AMG) preconditioner embedded in the PETSc library and we have studied the performance on the calibration of specific parameters. We have conducted our analysis on modern HPC architectures, performing scalability tests on multi-core and multi-GPUs settings. The results have shown that, for our problem, although scalability is verified on CPUs, GPUs are the optimal choice, since they yield the best performance in terms of solution time.

Accelerating Pythonic Coupled-Cluster Implementations: A Comparison Between CPUs and GPUs.

In this work, we benchmark several Python routines for time and memory requirements to identify the optimal choice of the tensor contraction operations available. We scrutinize how to accelerate the bottleneck tensor operations of Pythonic coupled-cluster implementations in the Cholesky linear algebra domain, utilizing a NVIDIA Tesla V100S PCIe 32GB (rev 1a) graphics processing unit (GPU). The NVIDIA compute unified device architecture API interacts with CuPy, an open-source library for Python, designed as a NumPy drop-in replacement for GPUs. Due to the limitations of video memory, the GPU calculations must be performed batch-wise. Timing results of some contractions containing large tensors are presented. The CuPy implementation leads to a factor of 10-16 speed-up of the bottleneck tensor contractions compared to computations on 36 central processing unit (CPU) cores. Finally, we compare example CCSD and pCCD-LCCSD calculations performed solely on CPUs to their CPU-GPU hybrid implementation, which leads to a speed-up of a factor of 3-4 compared to the CPU-only variant.

Hybrid CPU-GPU solution to regularized divergence-free curl-curl equations for electromagnetic inversion problems

The Curl-Curl equation is the foundation of time-harmonic electromagnetic (EM) problems in geophysics. The efficiency of its solution is key to EM simulations, accounting for over 95% of the computation cost in geophysical inversions for magnetotelluric or controlled-source EM problems. However, most published EM inversion codes are still central processing unit (CPU)-based and cannot utilize recent computational developments on the graphic processing units (GPUs). Based on a previously proposed divergence-free algorithm developed on CPUs, this study demonstrates the current limits of the CPU-based inversion procedure. To exploit the high throughput capability of GPUs, we propose a hybrid CPU-GPU framework to solve forward and adjoint problems required for EM inversions. The large sparse linear systems arising from the staggered-grid finite difference approximation of the Curl-Curl equation are solved with a mixed-precision Krylov subspace solver implemented on a GPU. The algorithm is then tested in EM forward and adjoint calculations, with real-world three-dimensional numerical examples. Test results show promising 30× kernel-level speed-ups over the conventional CPU algorithm. This approach may further take the complex frequency domain EM inversions onto the next, practical stage on small affordable GPU platforms.

Optimizing MRI Data Processing by exploiting GPU Acceleration for Efficient Image Analysis and Reconstruction

Magnetic Resonance Imaging (MRI) is a non-invasive medical imaging modality, offering detailed anatomical insights without ionizing radiation The advent of Graphics Processing Units (GPUs) has stimulated a paradigm shift, propelling MRI techniques to new frontiers. Through parallel processing capabilities, GPUs have expedited real-time imaging, complex image reconstruction, noise reduction, and intricate data analysis. This paper provides an extensive survey on the GPUs based MRI techniques and its synergistic impact on core methodologies such as Diffusion Tensor Imaging (DTI) and Functional MRI (fMRI). Through parallel processing and frameworks like CUDA and OpenCL, GPUs have overcome computational hurdles in MRI data processing. Challenges like memory constraints and data transfer bottlenecks are addressed through hybrid CPU-GPU strategies and algorithmic enhancements. The integration of GPUs yields faster scans, enhanced image quality, and real-time insights, benefiting patient care and accelerating medical research. Considering the ethical issues regarding patient data privacy and algorithmic fairness, GPUs' potential in MRI research and development is evident. This paper concludes by looking ahead about the future of GPU based MRI, urging further exploration to uncover new possibilities and shape a transformative path for the future of medical imaging.

Hybridhadoop: CPU-GPU hybrid scheduling in hadoop

Hybridhadoop: CPU-GPU hybrid scheduling in hadoop

Enhancing Adaptive Physics Refinement Simulations Through the Addition of Realistic Red Blood Cell Counts.

Simulations of cancer cell transport require accurately modeling mm-scale and longer trajectories through a circulatory system containing trillions of deformable red blood cells, whose intercellular interactions require submicron fidelity. Using a hybrid CPU-GPU approach, we extend the advanced physics refinement (APR) method to couple a finely-resolved region of explicitly-modeled red blood cells to a coarsely-resolved bulk fluid domain. We further develop algorithms that: capture the dynamics at the interface of differing viscosities, maintain hematocrit within the cell-filled volume, and move the finely-resolved region and encapsulated cells while tracking an individual cancer cell. Comparison to a fully-resolved fluid-structure interaction model is presented for verification. Finally, we use the advanced APR method to simulate cancer cell transport over a mm-scale distance while maintaining a local region of RBCs, using a fraction of the computational power required to run a fully-resolved model.

Accelerating Coupled-Cluster Calculations with GPUs: An Implementation of the Density-Fitted CCSD(T) Approach for Heterogeneous Computing Architectures Using OpenMP Directives.

An algorithm is presented for the coupled-cluster singles, doubles, and perturbative triples correction [CCSD(T)] method based on the density fitting or the resolution-of-the-identity (RI) approximation for performing calculations on heterogeneous computing platforms composed of multicore CPUs and graphics processing units (GPUs). The directive-based approach to GPU offloading offered by the OpenMP application programming interface has been employed to adapt the most compute-intensive terms in the RI-CCSD amplitude equations with computational costs scaling as , , and (where NO and NV denote the numbers of correlated occupied and virtual orbitals, respectively) and the perturbative triples correction to execute on GPU architectures. The pertinent tensor contractions are performed using an accelerated math library such as cuBLAS or hipBLAS. Optimal strategies are discussed for splitting large data arrays into tiles to fit them into the relatively small memory space of the GPUs, while also minimizing the low-bandwidth CPU-GPU data transfers. The performance of the hybrid CPU-GPU RI-CCSD(T) code is demonstrated on pre-exascale supercomputers composed of heterogeneous nodes equipped with NVIDIA Tesla V100 and A100 GPUs and on the world's first exascale supercomputer named "Frontier", the nodes of which consist of AMD MI250X GPUs. Speedups within the range 4-8× relative to the recently reported CPU-only algorithm are obtained for the GPU-offloaded terms in the RI-CCSD amplitude equations. Applications to polycyclic aromatic hydrocarbons containing 16-66 carbon atoms demonstrate that the acceleration of the hybrid CPU-GPU code for the perturbative triples correction relative to the CPU-only code increases with the molecule size, attaining a speedup of 5.7× for the largest circumovalene molecule (C66H20). The GPU-offloaded code enables the computation of the perturbative triples correction for the C60 molecule using the cc-pVDZ/aug-cc-pVTZ-RI basis sets in 7 min on Frontier when using 12,288 AMD GPUs with a parallel efficiency of 83.1%.

On floating point precision in computational fluid dynamics using OpenFOAM

Thanks to the computational power of modern cluster machines, numerical simulations can provide, with an unprecedented level of details, new insights into fluid mechanics. However, taking full advantage of this hardware remains challenging since data communication remains a significant bottleneck to reaching peak performances. Reducing floating point precision is a simple and effective way to reduce data movement and improve the computational speed of most applications. Nevertheless, special care needs to be taken to ensure the quality and convergence of computed solutions, especially when dealing with complex fluid simulations. In this work, we analyse the impact of reduced (single and mixed compared to double) precision on computational performance and accuracy for computational fluid dynamics. Using the open source library OpenFOAM, we consider incompressible, compressible, and multiphase fluid solvers for testing on relevant benchmarks for flows in the laminar and turbulent regime and in the presence of shock waves. Computational gain and changes in the scalability of applications in reduced precision are also discussed. In particular, an ad hoc theoretical model for the strong scaling allows us to interpret and understand the observed behaviours, as a function of floating point precision and hardware specifics. Finally, we show how reduced precision can significantly speed up a hybrid CPU–GPU implementation, made available to OpenFOAM end-users recently, that simply relies on a GPU linear algebra solver developed by hardware vendors.

Heterogeneous programming using OpenMP and CUDA/HIP for hybrid CPU-GPU scientific applications

Hybrid computer systems combine compute units (CUs) of different nature like CPUs, GPUs and FPGAs. Simultaneously exploiting the computing power of these CUs requires a careful decomposition of the applications into balanced parallel tasks according to both the performance of each CU type and the communication costs among them. This paper describes the design and implementation of runtime support for OpenMP hybrid GPU-CPU applications, when mixed with GPU-oriented programming models (e.g. CUDA/HIP). The paper describes the case for a hybrid multi-level parallelization of the NPB-MZ benchmark suite. The implementation exploits both coarse-grain and fine-grain parallelism, mapped to compute units of different nature (GPUs and CPUs). The paper describes the implementation of runtime support to bridge OpenMP and HIP, introducing the abstractions of Computing Unit and Data Placement. We compare hybrid and non-hybrid executions under state-of-the-art schedulers for OpenMP: static and dynamic task schedulings. Then, we improve the set of schedulers with two additional variants: a memorizing-dynamic task scheduling and a profile-based static task scheduling. On a computing node composed of one AMD EPYC 7742 @ 2.250 GHz (64 cores and 2 threads/core, totalling 128 threads per node) and 2 × GPU AMD Radeon Instinct MI50 with 32 GB, hybrid executions present speedups from 1.10× up to 3.5× with respect to a non-hybrid GPU implementation, depending on the number of activated CUs.

Optimizing Voronoi-based quantifications for reaching interactive analysis of 3D localizations in the million range.

Over the last decade, single-molecule localization microscopy (SMLM) has revolutionized cell biology, making it possible to monitor molecular organization and dynamics with spatial resolution of a few nanometers. Despite being a relatively recent field, SMLM has witnessed the development of dozens of analysis methods for problems as diverse as segmentation, clustering, tracking or colocalization. Among those, Voronoi-based methods have achieved a prominent position for 2D analysis as robust and efficient implementations were available for generating 2D Voronoi diagrams. Unfortunately, this was not the case for 3D Voronoi diagrams, and existing methods were therefore extremely time-consuming. In this work, we present a new hybrid CPU-GPU algorithm for the rapid generation of 3D Voronoi diagrams. Voro3D allows creating Voronoi diagrams of datasets composed of millions of localizations in minutes, making any Voronoi-based analysis method such as SR-Tesseler accessible to life scientists wanting to quantify 3D datasets. In addition, we also improve ClusterVisu, a Voronoi-based clustering method using Monte-Carlo simulations, by demonstrating that those costly simulations can be correctly approximated by a customized gamma probability distribution function.

A Review of High-Performance Computing Methods for Power Flow Analysis

Power flow analysis is critical for power systems due to the development of multiple energy supplies. For safety, stability, and real-time response in grid operation, grid planning, and analysis of power systems, it requires designing high-performance computing methods, accelerating power flow calculation, obtaining the voltage magnitude and phase angle of buses inside the power system, and coping with the increasingly complex large-scale power system. This paper provides an overview of the available parallel methods to fix the issues. Specifically, these methods can be classified into three categories from a hardware perspective: multi-cores, hybrid CPU-GPU architecture, and FPGA. In addition, from the perspective of numerical computation, the power flow algorithm is generally classified into iterative and direct methods. This review paper introduces models of power flow and hardware computing architectures and then compares their performance in parallel power flow calculations depending on parallel numerical methods on different computing platforms. Furthermore, this paper analyzes the challenges and pros and cons of these methods and provides guidance on how to exploit the parallelism of future power flow applications.

A SCALABLE HYBRID CPU-GPU COMPUTATIONAL FRAMEWORK FOR A FINITE ELEMENT-BASED AIR QUALITY MODEL

We propose a scalable computational framework for the hybrid CPU-GPU implementation ofa traffic-induced and finite element-based air quality model. The hybrid computing paradigm we investigate consists in combining the CPU-based distributed-memory programming approach using Message Passing Interface (MPI) and a GPU programming model for the finite element numerical integration using Compute Unified Device Architecture (CUDA), a general purpose parallel computing platform released by NVIDIA Corporation and featured on its own GPUs. The scalability results obtained from numerical experiments on two major road traffic-induced air pollutants, namely the fine and inhalable particulate matter PM$_{2.5}$ and PM$_{10}$, are illustrated. These achievements, including speedup and efficiency analyses, support that this framework scales well up to 256 CPU cores used concurrently with GPUs from a hybrid computing system.

A hybrid CPU–GPU solver based on three-dimensional spectral Chebyshev technique for determining the dynamic behavior of thick sandwich panels

In this study, static and vibration behavior of a curved thick sandwich-structured composite composed of a honeycomb core layer and two face-sheets reinforced with carbon nanotubes is investigated. The governing equations are derived using three-dimensional elasticity equations and Hamilton’s principle. The problem domain is discretized following Gauss–Lobatto sampling approach. To numerically solve the governing equations, a spectral method based on Chebyshev polynomials is used and a CPU–GPU based hybrid solver is developed. The presented approach is validated by comparing the predicted natural frequencies and deformation of the structure to those obtained from finite element analysis. The results show that in-house CPU–GPU based spectral solution approach decreases the computational duration remarkably and predicts the dynamic and static behavior of the system accurately. Since composite materials offer great flexibility in tailoring the dynamic behavior of the structure, an optimization study is also performed to benefit from the developed computationally efficient solver. In this study, the CNT orientations are selected as design variables to maximize the fundamental frequency of the structure. Based on the analyzed cases for different thickness ratios, curvature amounts, volume content, and distributions of CNTs, it is shown that the vibration behavior of the structure can be significantly tailored.

Fine-grained heterogeneous parallel direct solver for finite element problems

The improvement of computational effectiveness is a vital issue in the field of large-scale finite element analysis. The performance is fundamentally determined by the efficiency of solving sparse linear system equations using the implicit finite element method. This paper presents a direct linear solver based on heterogeneous hybrid parallel computing on CPUs and GPUs. This can efficiently utilize computing resources of multiple devices to achieve performance improvement. Initially, we partition the elimination tree into several subtrees to accomplish the task decomposition. Based on this, we build a dynamic programming mathematical model to balance the computational load of the various devices. Then, we develop a numerical decomposition strategy by combining node parallelism and tree parallelism for the CPUs. In addition, efficient numerical decomposition is achieved on the GPU through batch processing and maximizing the overlap between computations and data transfers. Numerical experiments show that, compared with MKL PARDISO, the performance of numerical factorization can be improved by up to 10 times by using CPU and dual-path GPU hybrid calculations, and the computation time of simulation can be reduced by one-third for the multicondition analysis of Body In White and by 20% for the large-scale nonlinear finite element deformation analysis.

Energy-Aware Non-Preemptive Task Scheduling With Deadline Constraint in DVFS-Enabled Heterogeneous Clusters

Energy conservation of large data centers for high performance computing workloads, such as deep learning with Big Data, is of critical significance, where cutting down a few percent of electricity translates into million-dollar savings. This work studies energy conservation on emerging CPU-GPU hybrid clusters through dynamic voltage and frequency scaling (DVFS). We aim at minimizing the total energy consumption of processing a batch of offline tasks or a sequence of real-time tasks under deadline constraints. We derive a fast and accurate analytical model to compute the appropriate voltage/frequency setting for each task, and assign multiple tasks to the cluster with heuristic scheduling algorithms. In particular, our model stresses the nonlinear relationship between task execution time and processor speed for GPU-accelerated applications, for more accurately capturing real-world GPU energy consumption. In performance evaluation driven by real-world power measurement traces, our scheduling algorithm shows comparable energy savings to the theoretical upper bound. With a GPU scaling interval where analytically at most 36% of energy can be saved, we record 33-35% of energy savings. Our results are applicable to energy management on modern heterogeneous clusters.

High Performance Adaptive Physics Refinement to Enable Large-Scale Tracking of Cancer Cell Trajectory.

The ability to track simulated cancer cells through the circulatory system, important for developing a mechanistic understanding of metastatic spread, pushes the limits of today's supercomputers by requiring the simulation of large fluid volumes at cellular-scale resolution. To overcome this challenge, we introduce a new adaptive physics refinement (APR) method that captures cellular-scale interaction across large domains and leverages a hybrid CPU-GPU approach to maximize performance. Through algorithmic advances that integrate multi-physics and multi-resolution models, we establish a finely resolved window with explicitly modeled cells coupled to a coarsely resolved bulk fluid domain. In this work we present multiple validations of the APR framework by comparing against fully resolved fluid-structure interaction methods and employ techniques, such as latency hiding and maximizing memory bandwidth, to effectively utilize heterogeneous node architectures. Collectively, these computational developments and performance optimizations provide a robust and scalable framework to enable system-level simulations of cancer cell transport.