GPU-accelerated Platforms Research Articles

Virtual dosimetry study with three cone-beam breast computed tomography scanners using a fast GPU-based Monte Carlo code

Objective. To compare the dosimetric performance of three cone-beam breast computed tomography (BCT) scanners, using real-time Monte Carlo-based dose estimates obtained with the virtual clinical trials (VCT)-BREAST graphical processing unit (GPU)-accelerated platform dedicated to VCT in breast imaging. Approach. A GPU-based Monte Carlo (MC) code was developed for replicating in silico the geometric, x-ray spectra and detector setups adopted, respectively, in two research scanners and one commercial BCT scanner, adopting 80 kV, 60 kV and 49 kV tube voltage, respectively. Our cohort of virtual breasts included 16 anthropomorphic voxelized breast phantoms from a publicly available dataset. For each virtual patient, we simulated exams on the three scanners, up to a nominal simulated mean glandular dose of 5 mGy (primary photons launched, in the order of 1011–1012 per scan). Simulated 3D dose maps (recorded for skin, adipose and glandular tissues) were compared for the same phantom, on the three scanners. MC simulations were implemented on a single NVIDIA GeForce RTX 3090 graphics card. Main results. Using the spread of the dose distribution as a figure of merit, we showed that, in the investigated phantoms, the glandular dose is more uniform within less dense breasts, and it is more uniformly distributed for scans at 80 kV and 60 kV, than at 49 kV. A realistic virtual study of each breast phantom was completed in about 3.0 h with less than 1% statistical uncertainty, with 109 primary photons processed in 3.6 s computing time. Significance. We reported the first dosimetric study of the VCT-BREAST platform, a fast MC simulation tool for real-time virtual dosimetry and imaging trials in BCT, investigating the dose delivery performance of three clinical BCT scanners. This tool can be adopted to investigate also the effects on the 3D dose distribution produced by changes in the geometrical and spectrum characteristics of a cone-beam BCT scanner.

HongTu: Scalable Full-Graph GNN Training on Multiple GPUs

Full-graph training on graph neural networks (GNN) has emerged as a promising training method for its effectiveness. Full-graph training requires extensive memory and computation resources. To accelerate this training process, researchers have proposed employing multi-GPU processing. However the scalability of existing frameworks is limited as they necessitate maintaining the training data for every layer in GPU memory. To efficiently train on large graphs, we present HongTu, a scalable full-graph GNN training system running on GPU-accelerated platforms. HongTu stores vertex data in CPU memory and offloads training to GPUs. HongTu employs a memory-efficient full-graph training framework that reduces runtime memory consumption by using partition-based training and recomputation-caching-hybrid intermediate data management. To address the issue of increased host-GPU communication caused by duplicated neighbor access among partitions, HongTu employs a deduplicated communication framework that converts the redundant host-GPU communication to efficient inter/intra-GPU data access. Further, HongTu uses a cost model-guided graph reorganization method to minimize communication overhead. Experimental results on a 4XA100 GPU server show that HongTu effectively supports billion-scale full-graph GNN training while reducing host-GPU data communication by 25%-71%. Compared to the full-graph GNN system DistGNN running on 16 CPU nodes, HongTu achieves speedups ranging from 7.8X to 20.2X. For small graphs where the training data fits into the GPUs, HongTu achieves performance comparable to existing GPU-based GNN systems.

HoloKinect: Holographic 3D Video Conferencing.

Recent world events have caused a dramatic rise in the use of video conferencing solutions such as Zoom and FaceTime. Although 3D capture and display technologies are becoming common in consumer products (e.g., Apple iPhone TrueDepth sensors, Microsoft Kinect devices, and Meta Quest VR headsets), 3D telecommunication has not yet seen any appreciable adoption. Researchers have made great progress in developing advanced 3D telepresence systems, but often with burdensome hardware and network requirements. In this work, we present HoloKinect, an open-source, user-friendly, and GPU-accelerated platform for enabling live, two-way 3D video conferencing on commodity hardware and a standard broadband internet connection. A Microsoft Azure Kinect serves as the capture device and a Looking Glass Portrait multiscopically displays the final reconstructed 3D mesh for a hologram-like effect. HoloKinect packs color and depth information into a single video stream, leveraging multiwavelength depth (MWD) encoding to store depth maps in standard RGB video frames. The video stream is compressed with highly optimized and hardware-accelerated video codecs such as H.264. A search of the depth and video encoding parameter space was performed to analyze the quantitative and qualitative losses resulting from HoloKinect's lossy compression scheme. Visual results were acceptable at all tested bitrates (3-30 Mbps), while the best results were achieved with higher video bitrates and full 4:4:4 chroma sampling. RMSE values of the recovered depth measurements were low across all settings permutations.

GPU-Accelerated Partially Linear Multiuser Detection for 5G and Beyond URLLC Systems

In this feasibility study, we have implemented a recently proposed partially linear multiuser detection algorithm in reproducing kernel Hilbert spaces (RKHSs) on a GPU-accelerated platform. Partially linear multiuser detection, which combines the robustness of linear detection with the power of nonlinear methods, has been proposed for a massive connectivity scenario with the non-orthogonal multiple access (NOMA). This is a promising approach, but detecting payloads within a received orthogonal frequency-division multiplexing (OFDM) radio frame requires the execution of a large number of inner product operations, which are the main computational burden of the algorithm. Although inner-product operations consist of simple kernel evaluations, their vast number poses a challenge in ultra-low latency (ULL) applications, because the time needed for computing the inner products might exceed the sub-millisecond latency requirement. To address this problem, this study demonstrates the acceleration of the inner-product operations through massive parallelization. The result is a GPU-accelerated real-time OFDM receiver that enables sub-millisecond latency detection to meet the requirements of 5th generation (5G) and beyond ultra-reliable and low latency communications (URLLC) systems. Moreover, the parallelization and acceleration techniques explored and demonstrated in this study can be extended to many other signal processing algorithms in Hilbert spaces, such as those based on projection onto convex sets (POCS) and adaptive projected subgradient method (APSM) algorithms. Experimental results and comparisons with the state-of-art confirm the effectiveness of our techniques.

GPU algorithms for Efficient Exascale Discretizations

In this paper we describe the research and development activities in the Center for Efficient Exascale Discretization within the US Exascale Computing Project, targeting state-of-the-art high-order finite-element algorithms for high-order applications on GPU-accelerated platforms. We discuss the GPU developments in several components of the CEED software stack, including the libCEED, MAGMA, MFEM, libParanumal, and Nek projects. We report performance and capability improvements in several CEED-enabled applications on both NVIDIA and AMD GPU systems.

Porting WarpX to GPU-accelerated platforms

WarpX is a general purpose electromagnetic particle-in-cell code that was originally designed to run on many-core CPU architectures. We describe the strategy, based on the AMReX library, followed to allow WarpX to use the GPU-accelerated nodes on OLCF’s Summit supercomputer, a strategy we believe will extend to the upcoming machines Frontier and Aurora. We summarize the challenges encountered, lessons learned, and give current performance results on a series of relevant benchmark problems.

Compiler-based code generation and autotuning for geometric multigrid on GPU-accelerated supercomputers

Generate parallel CUDA code from sequential C input code using a compiler-based tool for key operators in Geometric Multigrid.Demonstrate that generated code matched or improves performance of expert written CUDA code.Compare performance of generated code against a analytic performance bound derived from the Roofline model.Show weak scaling results of the generated code on the Titan supercomputer.Compares performance of Geometric Multigrid on the GPU based machines to CPU based machines. GPUs, with their high bandwidths and computational capabilities are an increasingly popular target for scientific computing. Unfortunately, to date, harnessing the power of the GPU has required use of a GPU-specific programming model like CUDA, OpenCL, or OpenACC. As such, in order to deliver portability across CPU-based and GPU-accelerated supercomputers, programmers are forced to write and maintain two versions of their applications or frameworks. In this paper, we explore the use of a compiler-based autotuning framework based on CUDA-CHiLL to deliver not only portability, but also performance portability across CPU- and GPU-accelerated platforms for the geometric multigrid linear solvers found in many scientific applications. We show that with autotuning we can attain near Roofline (a performance bound for a computation and target architecture) performance across the key operations in the miniGMG benchmark for both CPU- and GPU-based architectures as well as for a multiple stencil discretizations and smoothers. We show that our technology is readily interoperable with MPI resulting in performance at scale equal to that obtained via hand-optimized MPI+CUDA implementation.