Multiple Graphics Processing Units Research Articles

Accelerating 4D image reconstruction for magnetic resonance-guided radiotherapy.

Physiological motion impacts the dose delivered to tumours and vital organs in external beam radiotherapy and particularly in particle therapy. The excellent soft-tissue demarcation of 4D magnetic resonance imaging (4D-MRI) could inform on intra-fractional motion, but long image reconstruction times hinder its use in online treatment adaptation. Here we employ techniques from high-performance computing to reduce 4D-MRI reconstruction times below two minutes to facilitate their use in MR-guided radiotherapy. Four patients with pancreatic adenocarcinoma were scanned with a radial stack-of-stars gradient echo sequence on a 1.5T MR-Linac. Fast parallelised open-source implementations of the extra-dimensional golden-angle radial sparse parallel algorithm were developed for central processing unit (CPU) and graphics processing unit (GPU) architectures. We assessed the impact of architecture, oversampling and respiratory binning strategy on 4D-MRI reconstruction time and compared images using the structural similarity (SSIM) index against a MATLAB reference implementation. Scaling and bottlenecks for the different architectures were studied using multi-GPU systems. All reconstructed 4D-MRI were identical to the reference implementation (SSIM 0.99). Images reconstructed with overlapping respiratory bins were sharper at the cost of longer reconstruction times. The CPU +GPU implementation was over 17 times faster than the reference implementation, reconstructing images in 60 1 s and hyper-scaled using multiple GPUs. Respiratory-resolved 4D-MRI reconstruction times can be reduced using high-performance computing methods for online workflows in MR-guided radiotherapy with potential applications in particle therapy.

Phase-field lattice Boltzmann simulation of three-dimensional settling dendrite with natural convection during nonisothermal solidification of binary alloy

Understanding the motion and growth behaviors of equiaxed dendrites during solidification is important for predicting macrosegregation. In this study, we develop a phase-field lattice Boltzmann (PF-LB) simulation method for the settling and growth of an equiaxed dendrite during the nonisothermal solidification of a binary alloy. The PF-LB computations are accelerated by employing parallel computation using multiple graphic processing units (GPUs) and the octree block-structured adaptive mesh refinement method, which incorporates multiple mesh and time increment methods. By using the developed method, we can simulate the three-dimensional long-distance settling dendrite while considering the effects of latent heat release and natural convection. From the simulation results, we confirm that the natural convection due to the high solute concentration around a dendrite reduces the settling velocity. In addition, we observe that the temperature increase owing to latent heat release slows dendrite growth, which in turn slightly slows the settling velocity. From these results, we confirm that the effects of latent heat release and natural convection are not negligible in the quantitative evaluation of settling dendrites.

RTGPU: Real-Time GPU Scheduling of Hard Deadline Parallel Tasks With Fine-Grain Utilization

Many emerging cyber-physical systems, such as autonomous vehicles and robots, rely heavily on artificial intelligence and machine learning algorithms to perform important system operations. Since these highly parallel applications are computationally intensive, they need to be accelerated by graphics processing units (GPUs) to meet stringent timing constraints. However, despite the wide adoption of GPUs, efficiently scheduling multiple GPU applications while providing rigorous real-time guarantees remains a challenge. Each GPU application has multiple CPU execution and memory copy segments, with GPU kernels running on different hardware resources. Because of the complicated interactions between heterogeneous segments of parallel tasks, high schedulability is hard to achieve with conventional approaches. This paper proposes RTGPU, which combines fine-grain GPU partitioning on the system side with a novel scheduling algorithm on the theory side. Through system and theory co-design, RTGPU achieves superior system throughput and real-time schedulability. In this paper, we start by building a model for the CPU and memory copy segments. Leveraging persistent threads, we then implement fine-grained GPU partitioning with improved performance through interleaved execution. To reap the benefits of fine-grained GPU partitioning and schedule multiple parallel GPU applications, we propose a novel real-time scheduling algorithm based on federated scheduling and grid search with uniprocessor fixed-priority scheduling. Our approach provides real-time guarantees to meet hard deadlines, and achieves over 11% improvement in system throughput and up to 57% schedulability improvement compared with previous work. We validate and evaluate RTGPU on NVIDIA GTX1080Ti GPU systems. Our system side techniques can be applied on mainstream NVIDIA GPUs, and the proposed scheduling theory can be used in general heterogeneous computing platforms which have a similar task execution pattern.

Multi-Objective Neural Evolutionary Algorithm for Combinatorial Optimization Problems.

There has been a recent surge of success in optimizing deep reinforcement learning (DRL) models with neural evolutionary algorithms. This type of method is inspired by biological evolution and uses different genetic operations to evolve neural networks. Previous neural evolutionary algorithms mainly focused on single-objective optimization problems (SOPs). In this article, we present an end-to-end multi-objective neural evolutionary algorithm based on decomposition and dominance (MONEADD) for combinatorial optimization problems. The proposed MONEADD is an end-to-end algorithm that utilizes genetic operations and rewards signals to evolve neural networks for different combinatorial optimization problems without further engineering. To accelerate convergence, a set of nondominated neural networks is maintained based on the notion of dominance and decomposition in each generation. In inference time, the trained model can be directly utilized to solve similar problems efficiently, while the conventional heuristic methods need to learn from scratch for every given test problem. To further enhance the model performance in inference time, three multi-objective search strategies are introduced in this work. Our experimental results clearly show that the proposed MONEADD has a competitive and robust performance on a bi-objective of the classic travel salesman problem (TSP), as well as Knapsack problem up to 200 instances. We also empirically show that the designed MONEADD has good scalability when distributed on multiple graphics processing units (GPUs).

Space-to-speed architecture supporting acceleration on VHR image processing

One of the major focuses in the remote sensing community is the rapid processing of deep neural networks (DNNs) on very high resolution (VHR) aerial images. Few studies have investigated the acceleration of training and prediction by optimizing the architecture of the DNN system rather than designing a lightweight DNN. Parallel processing using multiple graphics processing units (GPUs) increases VHR image processing performance. It drives extremely large and frequent data transfers (input/output(I/O)) from random access memory (RAM) to GPU memory. As a result, the system bus congestion causes the system to hang, resulting in long latency in training/predicting. In this paper, we evaluate the causes of long latency and propose a space-to-speed (S2S) DNN system to overcome the aforementioned challenges. A three-level memory system aiming to reduce data transfer during system operation is presented. Distribution optimization with parallel processing was used to accelerate the training. Training optimizations on VHR images (such as hot-zone searching and image/ground truth queues for data saving) were used to train the VHR images efficiently. Inference optimization was performed to speed up prediction in the release mode. To verify the efficiency of the proposed system, we used aerial image labeling from the Institut National de Recherche en Informatique et en Automatique (INRIA) and benchmarks from the Massachusetts Institute of Technology Aerial Imagery for Roof Segmentation (MITAIRS) to test the system performance and accuracy. Without the loss of accuracy, the S2S system improved prediction speed on the testing dataset by eight GPUs in a normal setting in both the INRIA dataset (from 534 to 72 s) and the MITAIRS dataset (818 to 120 s). With the prediction in half-float (using float-16 data), an 8-GPU parallel processing increased the speed to 38 s in the INRIA dataset and 83 s in the MITAIRS dataset. In a pressure test, our proposed system operated on 18,000 images with a size of 5000 × 5000 from 18.2 to 1.8 h with the prediction in full-float (using float-32 data) and 43 min with the prediction in half-float, increasing the speed by a factor of 9.78 and 25.3, respectively, when compared to system runs without optimization.

Accelerating the density-functional tight-binding method using graphical processing units.

Acceleration of the density-functional tight-binding (DFTB) method on single and multiple graphical processing units (GPUs) was accomplished using the MAGMA linear algebra library. Two major computational bottlenecks of DFTB ground-state calculations were addressed in our implementation: the Hamiltonian matrix diagonalization and the density matrix construction. The code was implemented and benchmarked on two different computer systems: (1) the SUMMIT IBM Power9 supercomputer at the Oak Ridge National Laboratory Leadership Computing Facility with 1-6 NVIDIA Volta V100 GPUs per computer node and (2) an in-house Intel Xeon computer with 1-2 NVIDIA Tesla P100 GPUs. The performance and parallel scalability were measured for three molecular models of 1-, 2-, and 3-dimensional chemical systems, represented by carbon nanotubes, covalent organic frameworks, and water clusters.

Efficient Knowledge Graph Embedding Training Framework with Multiple GPUs

When training a large-scale knowledge graph embedding (KGE) model with multiple graphics processing units (GPUs), the partition-based method is necessary for parallel training. However, existing partition-based training methods suffer from low GPU utilization and high input/output (10) overhead between the memory and disk. For a high 10 overhead between the disk and memory problem, we optimized the twice partitioning with fine-grained GPU scheduling to reduce the 10 overhead between the CPU memory and disk. For low GPU utilization caused by the GPU load imbalance problem, we proposed balanced partitioning and dynamic scheduling methods to accelerate the training speed in different cases. With the above methods, we proposed fine-grained partitioning KGE, an efficient KGE training framework with multiple GPUs. We conducted experiments on some benchmarks of the knowledge graph, and the results show that our method achieves speedup compared to existing framework on the training of KGE.

A container-based workflow for distributed training of deep learning algorithms in HPC clusters

Deep learning has been postulated as a solution for numerous problems in different branches of science. Given the resource-intensive nature of these models, they often need to be executed on specialized hardware such graphical processing units (GPUs) in a distributed manner. In the academic field, researchers get access to this kind of resources through High Performance Computing (HPC) clusters. This kind of infrastructures make the training of these models difficult due to their multi-user nature and limited user permission. In addition, different HPC clusters may possess different peculiarities that can entangle the research cycle (e.g., libraries dependencies). In this paper we develop a workflow and methodology for the distributed training of deep learning models in HPC clusters which provides researchers with a series of novel advantages. It relies on udocker as containerization tool and on Horovod as library for the distribution of the models across multiple GPUs. udocker does not need any special permission, allowing researchers to run the entire workflow without relying on any administrator. Horovod ensures the efficient distribution of the training independently of the deep learning framework used. Additionally, due to containerization and specific features of the workflow, it provides researchers with a cluster-agnostic way of running their models. The experiments carried out show that the workflow offers good scalability in the distributed training of the models and that it easily adapts to different clusters.

High-Performance Statistical Computing in the Computing Environments of the 2020s.

Technological advances in the past decade, hardware and software alike, have made access to high-performance computing (HPC) easier than ever. We review these advances from a statistical computing perspective. Cloud computing makes access to supercomputers affordable. Deep learning software libraries make programming statistical algorithms easy and enable users to write code once and run it anywhere-from a laptop to a workstation with multiple graphics processing units (GPUs) or a supercomputer in a cloud. Highlighting how these developments benefit statisticians, we review recent optimization algorithms that are useful for high-dimensional models and can harness the power of HPC. Code snippets are provided to demonstrate the ease of programming. We also provide an easy-to-use distributed matrix data structure suitable for HPC. Employing this data structure, we illustrate various statistical applications including large-scale positron emission tomography and ℓ1-regularized Cox regression. Our examples easily scale up to an 8-GPU workstation and a 720-CPU-core cluster in a cloud. As a case in point, we analyze the onset of type-2 diabetes from the UK Biobank with 200,000 subjects and about 500,000 single nucleotide polymorphisms using the HPC ℓ1-regularized Cox regression. Fitting this half-million-variate model takes less than 45 minutes and reconfirms known associations. To our knowledge, this is the first demonstration of the feasibility of penalized regression of survival outcomes at this scale.

3D inversion of vertical gravity gradient with multiple graphics processing units based on matrix compression

Among single-component inversions, vertical gravity gradient data have been proven to be sufficient to obtain reasonable density distribution. However, the increasing numbers of observations and model parameters inevitably produce a large-scale sensitivity matrix, causing problems such as large memory occupation and slow computations in the inversion. Aiming at solving these long-standing problems, we have introduced a matrix compression technique to reduce memory occupation and constructed a hybrid parallel algorithm for accelerating focusing inversion. We develop the source codes and the design pattern for the proposed algorithm using C language with open multiprocessing and compute unified device architecture for further improving algorithm feasibility and reusability. A graphical user interface and visualization tools also are developed using Python. In the tests with synthetic data produced by numerical simulation and real data from Vinton Dome, it is verified that the inversion can produce an accurate result of density values and object location. The performance analysis is conducted using different scales of data. The speedup increases with the increasing number of graphics processing units (GPUs) and model parameters, proving that the proposed parallel algorithm is scalable. We compare this algorithm with others without matrix compression to demonstrate how the compressed sensitivity matrix storing and accessing method influences inversion performance. More specifically, for an inversion with 100 × 100 × 20 cells, the average speed is approximately 6 s per iteration, with a maximum speedup of 36.94. Our algorithm has a short running time and high multiple-GPU parallel efficiency. Hence, the hybrid parallel algorithm is more powerful at implementing large-scale data calculations and could be used for solving computing problems in mass data inversions.

Enabling real-time adaptation of machine learning models at x-ray Free Electron Laser facilities with high-speed training optimized computational hardware

The emergence of novel computational hardware is enabling a new paradigm for rapid machine learning model training. For the Department of Energy’s major research facilities, this developing technology will enable a highly adaptive approach to experimental sciences. In this manuscript we present the per-epoch and end-to-end training times for an example of a streaming diagnostic that is planned for the upcoming high-repetition rate x-ray Free Electron Laser, the Linac Coherent Light Source-II. We explore the parameter space of batch size and data parallel training across multiple Graphics Processing Units and Reconfigurable Dataflow Units. We show the landscape of training times with a goal of full model retraining in under 15 min. Although a full from scratch retraining of a model may not be required in all cases, we nevertheless present an example of the application of emerging computational hardware for adapting machine learning models to changing environments in real-time, during streaming data acquisition, at the rates expected for the data fire hoses of accelerator-based user facilities.

The numerical evaluation of Slater integrals on graphics processing units.

This article presents SlaterGPU, a graphics processing unit (GPU) accelerated library that uses OpenACC to numerically compute Slater-type orbital (STO) integrals. The electron repulsion integrals (ERI) are computed under the RI approximation using the Coulomb potential of the Slater basis function. To fully realize the performance capabilities of modern GPUs, the Slater integrals are evaluated in mixed-precision, resulting in speedups for the ERIs of over 80×. Parallelization on multiple GPUs allows for integral throughput of over 3 million integrals per second. This places STO integral throughput within reach of single-threaded, conventional Gaussian integration schemes. To test the quality of the integrals, the fluorine exchange reaction barrier in fluoromethane was computed using heat-bath configuration interaction (HBCI). In addition, the singlet-triplet gap of cyclobutadiene was examined using HBCI in a triple- , polarized basis set. These benchmarks demonstrate the library's ability to generate the full set of integrals necessary for configuration interaction with up to functions in the auxiliary basis.

Parallel-GPU AMR implementation for phase-field lattice Boltzmann simulation of a settling dendrite

The motion of the equiaxed grains plays an important role in macrosegregation. To simulate the motion of three-dimensional (3D) equiaxed dendrites growing in an undercooled melt, a large computational domain with small grids is required. In this study, we investigated the morphology and velocity of the settling dendrite and the growth velocity of its primary arms by implementing parallel computing with multiple graphics processing units (GPUs) for the adaptive mesh refinement (AMR) method (parallel-GPU AMR). A multiple-grid (MG) method that assigns different grid sizes to each field was also applied to the parallel-GPU AMR. A large-scale phase-field lattice Boltzmann (PF-LB) parallel-GPU AMR simulation of the settling dendrite was performed. The parallel GPU AMR method developed for the 3D PF-LB solidification model can evaluate the growth and motion of 3D equiaxed dendrite settling in an undercooled melt. The results are achieved with a significant acceleration of the 3D PF-LB simulations of settling dendrite and using a significantly reduced number of GPUs required for the simulation.

Parallel GPU-accelerated adaptive mesh refinement on two-dimensional phase-field lattice Boltzmann simulation of dendrite growth

Simulations of dendritic solidification involving melt convection and solid motion usually require a considerably higher computational domain than the dendrite size, whose computational efficiency with a uniform mesh is extremely low. In this study, to accelerate those two-dimensional simulations using the phase-field and lattice Boltzmann (PF-LB) methods, we developed a parallel computing method with multiple graphics processing units (GPUs) for the adaptive mesh refinement (AMR) method with dynamic load balancing (parallel-GPU AMR). It was confirmed that parallel-GPU AMR simulations were faster than those with the uniform mesh when the number of grid points in the adaptive mesh was around 40% or less than those in the uniform mesh. We also demonstrate that the developed parallel-GPU AMR can greatly accelerate the PF-LB simulations of dendrite growth with melt convection and solid motion.

Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data

While the advances in synchrotron light sources, together with the development of focusing optics and detectors, allow nanoscale ptychographic imaging of materials and biological specimens, the corresponding experiments can yield terabyte-scale volumes of data that can impose a heavy burden on the computing platform. Although graphics processing units (GPUs) provide high performance for such large-scale ptychography datasets, a single GPU is typically insufficient for analysis and reconstruction. Several works have considered leveraging multiple GPUs to accelerate the ptychographic reconstruction. However, most of these works utilize only the Message Passing Interface to handle the communications between GPUs. This approach poses inefficiency for a hardware configuration that has multiple GPUs in a single node, especially while reconstructing a single large projection, since it provides no optimizations to handle the heterogeneous GPU interconnections containing both low-speed (e.g., PCIe) and high-speed links (e.g., NVLink). In this paper, we provide an optimized intranode multi-GPU implementation that can efficiently solve large-scale ptychographic reconstruction problems. We focus on the maximum likelihood reconstruction problem using a conjugate gradient (CG) method for the solution and propose a novel hybrid parallelization model to address the performance bottlenecks in the CG solver. Accordingly, we have developed a tool, called PtyGer (Ptychographic GPU(multiple)-based reconstruction), implementing our hybrid parallelization model design. A comprehensive evaluation verifies that PtyGer can fully preserve the original algorithm’s accuracy while achieving outstanding intranode GPU scalability.

Real-time interactive visualization of large networks on a tiled display system

This paper introduces a methodology for visualizing large real-world (social) network data on a high-resolution tiled display system. Advances in network drawing algorithms enabled real-time visualization and interactive exploration of large real-world networks. However, visualization on a typical desktop monitor remains challenging due to the limited amount of screen space and ever increasing size of real-world datasets.To solve this problem, we propose an integrated approach that employs state-of-the-art network visualization algorithms on a tiled display system consisting of multiple screens. Key to our approach is to use the machine’s graphics processing units (GPUs) to their fullest extent, in order to ensure an interactive setting with real-time visualization. To realize this, we extended a recent GPU-based implementation of a force-directed graph layout algorithm to multiple GPUs and combined this with a distributed rendering approach in which each graphics card in the tiled display system renders precisely the part of the network to be displayed on the monitors attached to it.Our evaluation of the approach on a 12-screen 25 megapixels tiled display system with three GPUs, demonstrates interactive performance at 60 frames per second for real-world networks with tens of thousands of nodes and edges. This constitutes a performance improvement of approximately 4 times over a single GPU implementation. All the software developed to implement our tiled visualization approach, including the multi-GPU network layout, rendering, display and interaction components, are made available as open-source software.

Autotuning GPU code for acceleration of CGH calculation

Computer-generated hologram (CGH) has a problem in that the calculation time is huge, we use a graphics processing unit (GPU) to enable fast calculation for CGH. Optimization of GPU calculation is studied to bring out the best performance of the GPU. We propose an autotuning method for GPU code that can be applied when the scene or calculation environment changes in CGH for practical use. We created a system that finds the optimal parameters by presenting three parameters for tuning in CGH. We applied autotuning to multiple objects and different GPUs and found optimal parameters for each. In addition, it was confirmed that the calculation time was 1.1 to 1.7 times faster by the autotuning.

An MPI+OpenACC-based PRM scalar advection scheme in the GRAPES model over a cluster with multiple CPUs and GPUs

A moisture advection scheme is an essential module of a numerical weather/climate model representing the horizontal transport of water vapor. The Piecewise Rational Method (PRM) scalar advection scheme in the Global/Regional Assimilation and Prediction System (GRAPES) solves the moisture flux advection equation based on PRM. Computation of the scalar advection involves boundary exchange, and computation of higher bandwidth requirements is complicated and time-consuming in GRAPES. Recently, Graphics Processing Units (GPUs) have been widely used to solve scientific and engineering computing problems owing to advancements in GPU hardware and related programming models such as CUDA/OpenCL and Open Accelerator (OpenACC). Herein, we present an accelerated PRM scalar advection scheme with Message Passing Interface (MPI) and OpenACC to fully exploit GPUs' power over a cluster with multiple Central Processing Units (CPUs) and GPUs, together with optimization of various parameters such as minimizing data transfer, memory coalescing, exposing more parallelism, and overlapping computation with data transfers. Results show that about 3.5 times speedup is obtained for the entire model running at medium resolution with double precision when comparing the scheme's elapsed time on a node with two GPUs (NVIDIA P100) and two 16-core CPUs (Intel Gold 6142). Further, results obtained from experiments of a higher resolution model with multiple GPUs show excellent scalability.

Increasing momentum-like factors: A method for reducing training errors on multiple GPUs

In distributed training, increasing batch size can improve parallelism, but it can also bring many difficulties to the training process and cause training errors. In this work, we investigate the occurrence of training errors in theory and train ResNet-50 on CIFAR-10 by using Stochastic Gradient Descent (SGD) and Adaptive moment estimation (Adam) while keeping the total batch size in the parameter server constant and lowering the batch size on each Graphics Processing Unit (GPU). A new method that considers momentum to eliminate training errors in distributed training is proposed. We define a Momentum-like Factor (MF) to represent the influence of former gradients on parameter updates in each iteration. Then, we modify the MF values and conduct experiments to explore how different MF values influence the training performance based on SGD, Adam, and Nesterov accelerated gradient. Experimental results reveal that increasing MFs is a reliable method for reducing training errors in distributed training. The analysis of convergent conditions in distributed training with consideration of a large batch size and multiple GPUs is presented in this paper.

Detecting and Visualizing Observation Hot-Spots in Massive Volunteer-Contributed Geographic Data across Spatial Scales Using GPU-Accelerated Kernel Density Estimation

Volunteer-contributed geographic data (VGI) is an important source of geospatial big data that support research and applications. A major concern on VGI data quality is that the underlying observation processes are inherently biased. Detecting observation hot-spots thus helps better understand the bias. Enabled by the parallel kernel density estimation (KDE) computational tool that can run on multiple GPUs (graphics processing units), this study conducted point pattern analyses on tens of millions of iNaturalist observations to detect and visualize volunteers’ observation hot-spots across spatial scales. It was achieved by setting varying KDE bandwidths in accordance with the spatial scales at which hot-spots are to be detected. The succession of estimated density surfaces were then rendered at a sequence of map scales for visual detection of hot-spots. This study offers an effective geovisualization scheme for hierarchically detecting hot-spots in massive VGI datasets, which is useful for understanding the pattern-shaping drivers that operate at multiple spatial scales. This research exemplifies a computational tool that is supported by high-performance computing and capable of efficiently detecting and visualizing multi-scale hot-spots in geospatial big data and contributes to expanding the toolbox for geospatial big data analytics.