CPU GPU Research Articles

Exploring data flow design and vectorization with oneAPI for streaming applications on CPU+GPU

In recent times, oneAPI has emerged as a competitive framework to optimize streaming applications on heterogeneous CPU+GPU architectures, since it provides portability and performance thanks to the SYCL programming language and efficient parallel libraries as oneTBB. However, this approach opens up a wealth of implementations alternatives in this type of applications: from how to design the data flow to how to exploit data parallelism. Choosing the best alternative is not trivial, so in this paper we analyze them and contribute with an analytical model based on queue theory that helps in the on-line selection of the alternative that maximizes the throughput and the occupancy of the CPU and GPU compute units. We explore the design space offered by: a) different APIs to define the data flow (parallel_pipeline and Flow Graph from oneTBB, and SYCL events from SYCL); b) alternative kernel implementations to express data parallelism (SYCL, AVX and std::simd); and c) the mapping of the kernels into the available computing resources (CPU cores and GPU). The results show that the std::simd library can be 1.54x faster, 3% more energy efficient, and requires 7.36x less programming effort than AVX, and that implementations that enable asynchronous offloading of tasks to the devices as those based on SYCL events and Flow Graph APIs outperform the other APIs, being up to 1.10x faster and up to 1.18x more energy efficient.

Swarm–Intelligence-Based Task Scheduling for Reliability Optimization of Integrated CPU–GPU Edge Platforms in Cyber–Physical–Social Systems

Swarm–Intelligence-Based Task Scheduling for Reliability Optimization of Integrated CPU–GPU Edge Platforms in Cyber–Physical–Social Systems

Parallel implementation and performance of super-resolution generative adversarial network turbulence models for large-eddy simulation

Super-resolution (SR) generative adversarial networks (GANs) are promising for turbulence closure in large-eddy simulation (LES) due to their ability to accurately reconstruct high-resolution data from low-resolution fields. Current model training and inference strategies are not sufficiently mature for large-scale, distributed calculations due to the computational demands and often unstable training of SR-GANs, which limits the exploration of improved model structures, training strategies, and loss-function definitions. Integrating SR-GANs into LES solvers for inference-coupled simulations is also necessary to assess their a posteriori accuracy, stability, and cost. We investigate parallelization strategies for SR-GAN training and inference-coupled LES, focusing on computational performance and reconstruction accuracy. We examine distributed data-parallel training strategies for hybrid CPU–GPU node architectures and the associated influence of low-/high-resolution subbox size, global batch size, and discriminator accuracy. Accurate predictions require training subboxes that are sufficiently large relative to the Kolmogorov length scale. Care should be placed on the coupled effect of training batch size, learning rate, number of training subboxes, and discriminator’s learning capabilities. We introduce a data-parallel SR-GAN training and inference library for heterogeneous architectures that enables exchange between the LES solver and SR-GAN inference at runtime. We investigate the predictive accuracy and computational performance of this arrangement with particular focus on the overlap (halo) size required for accurate SR reconstruction. Similarly, a posteriori parallel scaling for efficient inference-coupled LES is constrained by the SR subdomain size, GPU utilization, and reconstruction accuracy. Based on these findings, we establish guidelines and best practices to optimize resource utilization and parallel acceleration of SR-GAN turbulence model training and inference-coupled LES calculations while maintaining predictive accuracy.

A Survey on Heterogeneous CPU–GPU Architectures and Simulators

ABSTRACTHeterogeneous architectures are vastly used in various high performance computing systems from IoT‐based embedded architectures to edge and cloud systems. Although heterogeneous architectures with cooperation of CPUs and GPUs and unified address space are increasingly used, there are still a lot of open questions and challenges regarding the design of these architectures. For evaluation, validation and exploration of next generation of heterogeneous CPU–GPU architectures, it is essential to use unified heterogeneous simulators for analyzing the execution of CPU–GPU workloads. This article presents a systematic review on challenges of heterogeneous CPU–GPU architectures with covering a diverse set of literatures on each challenge. The main considered challenges are shared resource management, network interconnections, task scheduling, energy consumption, and programming model. In addition, in this article, the state‐of‐the‐art of heterogeneous CPU–GPU simulation platforms is reviewed. The structure and characteristics of five cycle‐accurate heterogeneous CPU–GPU simulators are described and compared. We perform comprehensive discussions on the methodologies and challenges of designing high performance heterogeneous architectures. Moreover, for developing efficient heterogeneous CPU–GPU simulators, some recommendations are presented.

A Comprehensive of CPU and GPU Performance and Applications in Autonomous Vehicles

The rapid advancement of autonomous vehicle technology over the past decade has significantly increased the complexity of intelligent transportation systems. This need is clearly reflected in the requirements for high-performance computing in autonomous vehicles — a requirement that artificial intelligence, machine learning and big data analytic have all come to meet through their integration. We need the CPU and GPU power to handle processing in real-time data, sensor fusion and decision-making. We investigate how the CPU and GPU perform in an autonomous driving scenario, concluding that these tasks are complementary to one another describing where each can be best used within a heterogeneous computing architecture. The research delves into how these computing chips can be best used under the demand for real-time processing, reliability and efficiency. The study is to better understand the techniques for improving CPU-GPU collaboration, and applies these new findings on performance-intensive tasks like image recognition or track construction. The technical part involves conducting driving scenarios in the real world on three different types of CPU and several GPU, with key metrics such as processing latency, power consumption and accuracy. The actual experimental results show the advantages of GPU parallel computing and deep learning, while CPU still has a good advantage in multi-tasking ability and logical calculation. The results are important for the implementation of future autonomous driving systems, highlighting heterogeneous computing architectures as a means to optimize and support safe, effective vehicle operations.

Performance Study of an MRI Motion-Compensated Reconstruction Program on Intel CPUs, AMD EPYC CPUs, and NVIDIA GPUs

Motion-compensated image reconstruction enables new clinical applications of Magnetic Resonance Imaging (MRI), but it relies on computationally intensive algorithms. This study focuses on the Generalized Reconstruction by Inversion of Coupled Systems (GRICS) program, applied to the reconstruction of 3D images in cases of non-rigid or rigid motion. It uses hybrid parallelization with the MPI (Message Passing Interface) and OpenMP (Open Multi-Processing). For clinical integration, the GRICS needs to efficiently harness the computational resources of compute nodes. We aim to improve the GRICS’s performance without any code modification. This work presents a performance study of GRICS on two CPU architectures: Intel Xeon Gold and AMD EPYC. The roofline model is used to study the software–hardware interaction and quantify the code’s performance. For CPU–GPU comparison purposes, we propose a preliminary MATLAB–GPU implementation of the GRICS’s reconstruction kernel. We establish the roofline model of the kernel on two NVIDIA GPU architectures: Quadro RTX 5000 and A100. After the performance study, we propose some optimization patterns for the code’s execution on CPUs, first considering only the OpenMP implementation using thread binding and affinity and appropriate architecture-compilation flags and then looking for the optimal combination of MPI processes and OpenMP threads in the case of the hybrid MPI–OpenMP implementation. The results show that the GRICS performed well on the AMD EPYC CPUs, with an architectural efficiency of 52%. The kernel’s execution was fast on the NVIDIA A100 GPU, but the roofline model reported low architectural efficiency and utilization.

MicroMagnetic.jl: A Julia package for micromagnetic and atomistic simulations with GPU support

Abstract MicroMagnetic.jl is an open-source Julia package for micromagnetic and atomistic simulations. Using the features of the Julia programming language, MicroMagnetic.jl supports CPU and various GPU platforms, including NVIDIA, AMD, Intel, and Apple GPUs. Moreover, MicroMagnetic.jl supports Monte Carlo simulations for atomistic models and implements the nudged-elastic-band method for energy barrier computations. With built-in support for double and single precision modes and a design allowing easy extensibility to add new features, MicroMagnetic.jl provides a versatile toolset for researchers in micromagnetics and atomistic simulations.

Anatomizing Deep Learning Inference in Web Browsers

Web applications have increasingly adopted Deep Learning (DL) through in-browser inference , wherein DL inference performs directly within Web browsers. The actual performance of in-browser inference and its impacts on the quality of experience ( QoE ) remain unexplored, and urgently require new QoE measurements beyond traditional ones, e.g., mainly focusing on page load time. To bridge this gap, we make the first comprehensive performance measurement of in-browser inference to date. Our approach proposes new metrics to measure in-browser inference: responsiveness, smoothness, and inference accuracy. Our extensive analysis involves 9 representative DL models across Web browsers of 50 popular PC devices and 20 mobile devices. The results reveal that in-browser inference exhibits a substantial latency gap, averaging 16.9 times slower on CPU and 4.9 times slower on GPU compared to native inference on PC devices. The gap on mobile CPU and mobile GPU is 15.8 times and 7.8 times, respectively. Furthermore, we identify contributing factors to such latency gap, including underutilized hardware instruction sets, inherent overhead in the runtime environment, resource contention within the browser, and inefficiencies in software libraries and GPU abstractions. Additionally, in-browser inference imposes significant memory demands, at times exceeding 334.6 times the size of the DL models themselves, partly attributable to suboptimal memory management. We also observe that in-browser inference leads to a significant 67.2% increase in the time it takes for GUI components to render within Web browsers, significantly affecting the overall user QoE of Web applications reliant on this technology.

A high-performance dynamic scheduling for sparse matrix-based applications on heterogeneous CPU–GPU environment

A high-performance dynamic scheduling for sparse matrix-based applications on heterogeneous CPU–GPU environment

Adapting arepo-rt for exascale computing: GPU acceleration and efficient communication

ABSTRACT Radiative transfer (RT) is a crucial ingredient for self-consistent modelling of numerous astrophysical phenomena across cosmic history. However, on-the-fly integration into radiation hydrodynamics (RHD) simulations is computationally demanding, particularly due to the stringent time-stepping conditions and increased dimensionality inherent in multifrequency collisionless Boltzmann physics. The emergence of exascale supercomputers, equipped with extensive CPU cores and GPU accelerators, offers new opportunities for enhancing RHD simulations. We present the first steps towards optimizing arepo-rt for such high-performance computing environments. We implement a novel node-to-node (n-to-n) communication strategy that utilizes shared memory to substitute intranode communication with direct memory access. Furthermore, combining multiple internode messages into a single message substantially enhances network bandwidth utilization and performance for large-scale simulations on modern supercomputers. The single-message n-to-n approach also improves performance on smaller scale machines with less optimized networks. Furthermore, by transitioning all RT-related calculations to GPUs, we achieve a significant computational speedup of around 15 for standard benchmarks compared to the original CPU implementation. As a case study, we perform cosmological RHD simulations of the Epoch of Reionization, employing a similar setup as the THESAN project. In this context, RT becomes sub-dominant such that even without modifying the core arepo codebase, there is an overall threefold improvement in efficiency. The advancements presented here have broad implications, potentially transforming the complexity and scalability of future simulations for a wide variety of astrophysical studies. Our work serves as a blueprint for porting similar simulation codes based on unstructured resolution elements to GPU-centric architectures.

Modeling of the fracture behaviors of concrete using 3D discrete element method with softening effect

Understanding the fracture mechanism of concrete is very important for the safety design of concrete structures. We develop a 3D discrete element model with softening effect to study the fracture behaviors of quasi-brittle materials. The basic ideas of our model inspired from the flat-joint concept and the cohesive crack model are implemented in the framework of MUSEN, which is an open source discrete element software with CPU–GPU accelerating computation. The placement-package-compaction method is proposed together with laser scanning technology to construct the meso-scale geometric model of concrete. The meso-parameters of 3D discrete element model developed are calibrated by uniaxial compression and splitting tension tests of concrete. The validation of the present model is verified by the laboratory test of concrete. Compared with the results predicted by the flat-joint model, the F-CMOD curves and fracture parameters simulated by our model are in good agreement with that of the laboratory tests of concrete. Finally, it is concluded that the fracture strain of bond significantly affects the fracture behavior but the softening mode does not. It is found that the linear one is better choice among three softening modes in the present model.

Maboss for HPC environments: implementations of the continuous time Boolean model simulator for large CPU clusters and GPU accelerators

BackgroundComputational models in systems biology are becoming more important with the advancement of experimental techniques to query the mechanistic details responsible for leading to phenotypes of interest. In particular, Boolean models are well fit to describe the complexity of signaling networks while being simple enough to scale to a very large number of components. With the advance of Boolean model inference techniques, the field is transforming from an artisanal way of building models of moderate size to a more automatized one, leading to very large models. In this context, adapting the simulation software for such increases in complexity is crucial.ResultsWe present two new developments in the continuous time Boolean simulators: MaBoSS.MPI, a parallel implementation of MaBoSS which can exploit the computational power of very large CPU clusters, and MaBoSS.GPU, which can use GPU accelerators to perform these simulations.ConclusionThese implementations enable simulation and exploration of the behavior of very large models, thus becoming a valuable analysis tool for the systems biology community.

Allok: a machine learning approach for efficient graph execution on CPU–GPU clusters

Allok: a machine learning approach for efficient graph execution on CPU–GPU clusters

CPU–GPU heterogeneous code acceleration of a finite volume Computational Fluid Dynamics solver

This research focuses on accelerating the finite-volume Computational Fluid Dynamics (CFD) solver, SENSEI, through concurrent CPU–GPU heterogeneous computing, leveraging multiple CPUs and GPUs. An overview of SENSEI, its discretization, and the heterogeneous computing workflow utilizing MPI and OpenACC are provided. A performance model for CPU–GPU heterogeneous computing, incorporating ghost cell exchange usually applied in stencil computations, is introduced to estimate the performance. This work explores the scaling performance of CPU–GPU heterogeneous computing in comparison to a pure multi-CPU/GPU setup using a supersonic inlet test case. The results emphasize the advantages of leveraging both CPU and GPU computational power concurrently. Highlighting the collaborative synergy of CPUs and GPUs as co-workers, this work shows improved performance over exclusive use of pure CPUs or GPUs, with the proposed performance model ensuring a fair estimation of these advantages. This work concludes by offering suggestions for application users interested in accelerating scientific computing simulations and providing feedback for hardware architects designing improved CPU–GPU heterogeneous systems for heterogeneous computing.

Physical mechanism-corrected degradation trend prediction network under data missing

Accurate degradation trend prediction (DTP) is crucial for optimizing equipment operation and maintenance, thereby boosting production efficiency. This study introduces a novel Data Repair and Dual-data-stream LSTM (DR-DLSTM) network to tackle the challenge of missing data in equipment DTP. The proposed DR-DLSTM framework employs convex optimization to consider both the trend and periodic variations in the data, incorporating polynomial and trigonometric functions into the implicit feature matrix to construct latent vectors for missing data rectification. The network features a Dual-LSTM block with dual data streams to enhance feature extraction, with two gating update units correlating time series components and redistributing feature weights. The Dual-LSTM enables separate and accurate prediction of trend and periodic components, thereby enhancing the feature extraction capability of the prediction model. Additionally, the integration of physical rule information through Fourier and wavelet transform frequency correction modules allows for dynamic adjustments in prediction outcomes, from global trends to localized details. The DR-DLSTM's effectiveness is demonstrated through comprehensive comparisons with state-of-the-art models across multiple datasets, highlighting its superior performance. The results demonstrate the superiority of the proposed model. These algorithms were implemented in Python using Torch on a 2.9 GHz Intel I7 CPU and TITAN Xp GPU.

CPU–GPU Heterogeneous Computation Offloading and Resource Allocation Scheme for Industrial Internet of Things

CPU–GPU Heterogeneous Computation Offloading and Resource Allocation Scheme for Industrial Internet of Things

Direct reduction of iron-ore with hydrogen in fluidized beds: A coarse-grained CFD-DEM-IBM study

Hydrogen metallurgy technology uses hydrogen as the reducing agent instead of carbon reduction, which is one of the important ways to reduce carbon dioxide emissions and ensure the green and sustainable development of iron and steel industry. Due to the advantages of high gas-solid contact efficiency and outstanding mass and heat transfer, direct reduction of iron ore in fluidized beds has attracted much attention. Based on the three-layer unreacted shrinking core model and considering the diversity of particle structure during the reaction process, a multi-reaction path (multi-resistance network) model was established. In this study, a coarse-grained CFD-DEM-IBM solver based on hybrid CPU–GPU computing is developed to simulate the direct reduction process of two kinds of iron ore with hydrogen in fluidized beds, where the developed unreacted shrinking core model is used to model the reduction reactions, a coarse-grained model and multiple GPUs enable the significant acceleration of particle computation, and the immersed boundary method (IBM) enables the use of simple mesh even in complex geometries of reactors. The predicted results of particle reduction degree are in good agreement with the experimental values, which proves the correctness of the CFD-DEM-IBM solver. In addition, the effects of reaction kinetic parameters and operating temperature on particle reduction degree are also investigated. Present study provides a method for digital design, optimization and scale-up of ironmaking reactors.

A comparison of Algebraic Multigrid Bidomain solvers on hybrid CPU–GPU architectures

The numerical simulation of cardiac electrophysiology is a highly challenging problem in scientific computing. The Bidomain system is the most complete mathematical model of cardiac bioelectrical activity. It consists of an elliptic and a parabolic partial differential equation (PDE), of reaction–diffusion type, describing the spread of electrical excitation in the cardiac tissue. The two PDEs are coupled with a stiff system of ordinary differential equations (ODEs), representing ionic currents through the cardiac membrane. Developing efficient and scalable preconditioners for the linear systems arising from the discretization of such computationally challenging model is crucial in order to reduce the computational costs required by the numerical simulations of cardiac electrophysiology. In this work, focusing on the Bidomain system as a model problem, we have benchmarked two popular implementations of the Algebraic Multigrid (AMG) preconditioner embedded in the PETSc library and we have studied the performance on the calibration of specific parameters. We have conducted our analysis on modern HPC architectures, performing scalability tests on multi-core and multi-GPUs settings. The results have shown that, for our problem, although scalability is verified on CPUs, GPUs are the optimal choice, since they yield the best performance in terms of solution time.

Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses

Graph Neural Networks (GNNs) are emerging as a powerful tool for learning from graph-structured data and performing sophisticated inference tasks in various application domains. Although GNNs have been shown to be effective on modest-sized graphs, training them on large-scale graphs remains a significant challenge due to the lack of efficient storage access and caching methods for graph data. Existing frameworks for training GNNs use CPUs for graph sampling and feature aggregation, while the training and updating of model weights are executed on GPUs. However, our in-depth profiling shows CPUs cannot achieve the graph sampling and feature aggregation throughput required to keep up with GPUs. Furthermore, when the graph and its embeddings do not fit in the CPU memory, the overhead introduced by the operating system, say for handling page-faults, causes gross under-utilization of hardware and prolonged end-to-end execution time. To address these issues, we propose the GPU Initiated Direct Storage Access (GIDS) dataloader, to enable GPU-oriented GNN training for large-scale graphs while efficiently utilizing all hardware resources, such as CPU memory, storage, and GPU memory. The GIDS dataloader first addresses memory capacity constraints by enabling GPU threads to directly fetch feature vectors from storage. Then, we introduce a set of innovative solutions, including the dynamic storage access accumulator, constant CPU buffer, and GPU software cache with window buffering, to balance resource utilization across the entire system for improved end-to-end training throughput. Our evaluation using a single GPU on terabyte-scale GNN datasets shows that the GIDS dataloader accelerates the overall DGL GNN training pipeline by up to 582× when compared to the current, state-of-the-art DGL dataloader.

End‐to‐End Compressed Meshlet Rendering

AbstractIn this paper, we study rendering of end‐to‐end compressed triangle meshes using modern GPU techniques, in particular, mesh shaders. Our approach allows us to keep unstructured triangle meshes in GPU memory in compressed form and decompress them in shader code just in time for rasterization. Typical previous approaches use a compressed mesh format only for persistent storage and streaming, but must decompress it into GPU memory before submitting it to rendering. In contrast, our approach uses an identical compressed format in both storage and GPU memory. Hence, our compression method effectively reduces the in‐memory requirements of huge triangular meshes and avoids any waiting times on streaming geometry induced by the need for a decompression stage on the CPU. End‐to‐end compression also means that scenes with more geometric detail than previously possible can be made fully resident in GPU memory. Our approach is based on a novel decomposition of meshes into meshlets, i.e. disjoint primitive groups that are compressed individually. Decompression using a mesh shader allows de facto random access on the primitive level, which is important for applications such as selective streaming and fine‐grained visibility computation. We compare our approach to multiple commonly used compressed meshlet formats in terms of required memory and rendering times. The results imply that our approach reduces the required CPU–GPU memory bandwidth, a frequent bottleneck in out‐of‐core rendering.