Multiple Graphics Processing Units Research Articles

Regional-scale fault-to-structure earthquake simulations with the EQSIM framework: Workflow maturation and computational performance on GPU-accelerated exascale platforms

Continuous advancements in scientific and engineering understanding of earthquake phenomena, combined with the associated development of representative physics-based models, is providing a foundation for high-performance, fault-to-structure earthquake simulations. However, regional-scale applications of high-performance models have been challenged by the computational requirements at the resolutions required for engineering risk assessments. The EarthQuake SIMulation (EQSIM) framework, a software application development under the US Department of Energy (DOE) Exascale Computing Project, is focused on overcoming the existing computational barriers and enabling routine regional-scale simulations at resolutions relevant to a breadth of engineered systems. This multidisciplinary software development—drawing upon expertise in geophysics, engineering, applied math and computer science—is preparing the advanced computational workflow necessary to fully exploit the DOE’s exaflop computer platforms coming online in the 2023 to 2024 timeframe. Achievement of the computational performance required for high-resolution regional models containing upward of hundreds of billions to trillions of model grid points requires numerical efficiency in every phase of a regional simulation. This includes run time start-up and regional model generation, effective distribution of the computational workload across thousands of computer nodes, efficient coupling of regional geophysics and local engineering models, and application-tailored highly efficient transfer, storage, and interrogation of very large volumes of simulation data. This article summarizes the most recent advancements and refinements incorporated in the workflow design for the EQSIM integrated fault-to-structure framework, which are based on extensive numerical testing across multiple graphics processing unit (GPU)-accelerated platforms, and demonstrates the computational performance achieved on the world’s first exaflop computer platform through representative regional-scale earthquake simulations for the San Francisco Bay Area in California, USA.

Multi-phase-field lattice Boltzmann simulations of semi-solid simple shear deformation in thin film

Semi-solid deformation during casting often results in significant solidification defects, such as segregation bands. Consequently, the development of a numerical simulation tool is crucial for accurately replicating semi-solid deformation. In our previous study, we applied a multi-phase-field lattice Boltzmann (MPF-LB) model to semi-solid deformation, facilitating seamless simulation from polycrystalline solidification to semi-solid deformation in a two-dimensional (2D) problem. This study extends the 2D MPF-LB model to a three-dimensional (3D) problem and develops a simulation method for semi-solid simple shear deformation in thin films. To enhance the efficiency of the 3D semi-solid simulation, we implemented parallel computations using multiple graphics processing units. Through a discussion of the relationships among the stress–strain curve, grain rearrangement behavior, and fluid flow, we confirmed that the developed 3D MPF-LB model successfully reproduced the characteristic phenomena of semi-solid deformation, and has high potential to investigate the nuanced mechanisms of semi-solid deformation.

Localization of overdamped bosonic modes and transport in strange metals

The strange metal phase of correlated electrons materials was described in a recent theory by a model of a Fermi surface coupled a two-dimensional quantum critical bosonic field with a spatially random Yukawa coupling. With the assumption of self-averaging randomness, similar to that in the Sachdev-Ye-Kitaev model, numerous observed properties of a strange metal were obtained for a wide range of intermediate temperatures, including the linear in temperature resistivity. The Harris criterion implies that spatial fluctuations in the local position of the critical point must dominate at lower temperatures. For an [Formula: see text]-component boson with [Formula: see text], we use multiple graphics processing units (GPUs) to compute the real frequency spectrum of the boson propagator in a self-consistent mean-field treatment of the boson self-interactions, but an exact treatment of multiple realizations of the spatial randomness from the random boson mass. We find that Landau damping from the fermions leads to the emergence of the physics of the random transverse-field Ising model at low temperatures, as has been proposed by Hoyos, Kotabage, and Vojta. This regime is controlled by localized overdamped eigenmodes of the bosonic scalar field, also has a resistivity which is nearly linear-in-temperature, and extends into a "quantum critical phase" away from the quantum critical point, as observed in several cuprates. For the [Formula: see text] Ising scalar, the mean-field treatment is not applicable, and so we use Hybrid Monte Carlo simulations running on multiple GPUs; we find a rounded transition and localization physics, with strange metal behavior in an extended region around the transition.

Iterative Reconstruction of Micro Computed Tomography Scans Using Multiple Heterogeneous GPUs.

Graphics processing units (GPUs) facilitate massive parallelism and high-capacity storage, and thus are suitable for the iterative reconstruction of ultrahigh-resolution micro computed tomography (CT) scans by on-the-fly system matrix (OTFSM) calculation using ordered subsets expectation maximization (OSEM). We propose a finite state automaton (FSA) method that facilitates iterative reconstruction using a heterogeneous multi-GPU platform through parallelizing the matrix calculations derived from a ray tracing system of ordered subsets. The FSAs perform flow control for parallel threading of the heterogeneous GPUs, which minimizes the latency of launching ordered-subsets tasks, reduces the data transfer between the main system memory and local GPU memory, and solves the memory-bound of a single GPU. In the experiments, we compared the operation efficiency of OS-MLTR for three reconstruction environments. The heterogeneous multiple GPUs with job queues for high throughput calculation speed is up to five times faster than the single GPU environment, and that speed up is nine times faster than the heterogeneous multiple GPUs with the FIFO queues of the device scheduling control. Eventually, we proposed an event-triggered FSA method for iterative reconstruction using multiple heterogeneous GPUs that solves the memory-bound issue of a single GPU at ultrahigh resolutions, and the routines of the proposed method were successfully executed on each GPU simultaneously.

High-Performance Multi-GPU Analytic RI-MP2 Energy Gradients.

This article presents a novel algorithm for the calculation of analytic energy gradients from second-order Møller-Plesset perturbation theory within the Resolution-of-the-Identity approximation (RI-MP2), which is designed to achieve high performance on clusters with multiple graphical processing units (GPUs). The algorithm uses GPUs for all major steps of the calculation, including integral generation, formation of all required intermediate tensors, solution of the Z-vector equation and gradient accumulation. The implementation in the EXtreme Scale Electronic Structure System (EXESS) software package includes a tailored, highly efficient, multistream scheduling system to hide CPU-GPU data transfer latencies and allows nodes with 8 A100 GPUs to operate at over 80% of theoretical peak floating-point performance. Comparative performance analysis shows a significant reduction in computational time relative to traditional multicore CPU-based methods, with our approach achieving up to a 95-fold speedup over the single-node performance of established software such as Q-Chem and ORCA. Additionally, we demonstrate that pairing our implementation with the molecular fragmentation framework in EXESS can drastically lower the computational scaling of RI-MP2 gradient calculations from quintic to subquadratic, enabling further substantial savings in runtime while retaining high numerical accuracy in the resulting gradients.

Multi-Device Parallel MRI Reconstruction: Efficient Partitioning for Undersampled 5D Cardiac CINE.

Cardiac CINE, a form of dynamic cardiac MRI, is indispensable in the diagnosis and treatment of heart conditions, offering detailed visualization essential for the early detection of cardiac diseases. As the demand for higher-resolution images increases, so does the volume of data requiring processing, presenting significant computational challenges that can impede the efficiency of diagnostic imaging. Our research presents an approach that takes advantage of the computational power of multiple Graphics Processing Units (GPUs) to address these challenges. GPUs are devices capable of performing large volumes of computations in a short period, and have significantly improved the cardiac MRI reconstruction process, allowing images to be produced faster. The innovation of our work resides in utilizing a multi-device system capable of processing the substantial data volumes demanded by high-resolution, five-dimensional cardiac MRI. This system surpasses the memory capacity limitations of single GPUs by partitioning large datasets into smaller, manageable segments for parallel processing, thereby preserving image integrity and accelerating reconstruction times. Utilizing OpenCL technology, our system offers adaptability and cross-platform functionality, ensuring wider applicability. The proposed multi-device approach offers an advancement in medical imaging, accelerating the reconstruction process and facilitating faster and more effective cardiac health assessment.

PARALLEL AND DISTRIBUTED COMPUTING TECHNOLOGIES FOR AUTONOMOUS VEHICLE NAVIGATION

Context. Autonomous vehicles are becoming increasingly popular, and one of the important modern challenges in their development is ensuring their effective navigation in space and movement within designated lanes. This paper examines a method of spatial orientation for vehicles using computer vision and artificial neural networks. The research focused on the navigation system of an autonomous vehicle, which incorporates the use of modern distributed and parallel computing technologies.
 Objective. The aim of this work is to enhance modern autonomous vehicle navigation algorithms through parallel training of artificial neural networks and to determine the optimal combination of technologies and nodes of devices to increase speed and enable real-time decision-making capabilities in spatial navigation for autonomous vehicles.
 Method. The research establishes that the utilization of computer vision and neural networks for road lane segmentation proves to be an effective method for spatial orientation of autonomous vehicles. For multi-core computing systems, the application of parallel programming technology, OpenMP, for neural network training on processors with varying numbers of parallel threads increases the algorithm’s execution speed. However, the use of CUDA technology for neural network training on a graphics processing unit significantly enhances prediction speeds compared to OpenMP. Additionally, the feasibility of employing PyTorch Distributed Data Parallel (DDP) technology for training the neural network across multiple graphics processing units (nodes) simultaneously was explored. This approach further improved prediction execution times compared to using a single graphics processing unit.
 Results. An algorithm for training and prediction of an artificial neural network was developed using two independent nodes, each equipped with separate graphics processing units, and their synchronization for exchanging training results after each epoch, employing PyTorch Distributed Data Parallel (DDP) technology. This approach allows for scalable computations across a higher number of resources, significantly expediting the model training process.
 Conclusions. The conducted experiments have affirmed the effectiveness of the proposed algorithm, warranting the recommendation of this research for further advancement in autonomous vehicles and enhancement of their navigational capabilities. Notably, the research outcomes can find applications in various domains, encompassing automotive manufacturing, logistics, and urban transportation infrastructure. The obtained results are expected to assist future researchers in understanding the most efficient hardware and software resources to employ for implementing AI-based navigation systems in autonomous vehicles. Prospects for future investigations may encompass refining the accuracy of the proposed parallel algorithm without compromising its efficiency metrics. Furthermore, there is potential for experimental exploration of the proposed algorithm in more intricate practical scenarios of diverse nature and dimensions.

GPU-based transient analysis of modern grids deploying a hybrid DDM algorithm

Solving a significant number of differential-algebraic equations is essential for conducting transient stability simulations on a complex and interconnected electrical power system. Because of the increasing size and intricacy of modern grids, traditional sequential simulation methods are becoming more computationally challenging and time-consuming to use for dynamic simulations. This research aims to examine parallelism techniques from an innovative algorithm viewpoint to the hardware level by utilizing multiple graphics processing units as the main computational tool designed to simulate the dynamic behaviour of power systems on a large scale. Our approach features a novel hybrid algorithm to improve the dynamic simulations of grids with significant renewable energy integration. The algorithm employs a Schwarz-based approach in its initial decomposition phase to isolate synchronous machines and renewable sources from the grid. This separation enhances the algorithm's ability to manage these components effectively. Subsequently, each subsystem undergoes further division into subdomains in the second stage, with separate computations carried out using the Schur-complement technique. The simulations were conducted on various test systems, with a maximum of 25,000 buses, 8,000 synchronous generators, and 256 PV farms all modelled in detail. We introduce a GPU-oriented preprocessing and vectorization parallelization method for the implementation of our two-stage algorithm. This approach provides the flexibility to tailor parallelization techniques, including nested for loops and kernel pragmas, to craft a hardware solution that enhances the performance of our dynamic domain decomposition algorithm. Simulation outcomes have verified that this approach can yield a notable 7.8x acceleration in speed when executed on an NVIDIA GeForce RTX 2070 SUPER GPU.

Accelerating micromagnetic and atomistic simulations using multiple GPUs

It is shown that micromagnetic and atomistic spin dynamics simulations can use multiple graphical processing units (GPUs) not only to reduce computation time but also to allow for a larger simulation size than is possible on a single GPU. While interactions that depend on neighboring spins, such as exchange interactions, may be implemented efficiently by transferring data between GPUs using halo regions or direct memory accesses, implementing the long-range demagnetizing interaction is the main difficulty in achieving good performance scaling, where the data transfer rate between GPUs is a significant bottleneck. A multi-GPU convolution algorithm is developed here, which relies on single-GPU FFTs executed in parallel. It is shown that even for micromagnetic simulations where the demagnetizing interaction computation time dominates, good performance scaling may be achieved, with speedup factors up to 1.8, 2.5, and 3.1, for two, three, and four GPUs, respectively. The code developed here can be used for any number of GPUs in parallel, with performance scaling strongly dependent on the inter-GPU data transfer rate and connection topology. This is further improved in micromagnetic simulations, which include a spin transport solver, obtaining speedup factors up to 1.96, 2.8, and 3.7, for two, three, and four GPUs, respectively. The best case scenario is obtained for atomistic simulations, where the demagnetizing interaction is implemented with spin-averaged cells. Using a single workstation with four GPUs, it is shown that atomistic spin dynamics simulations with up to 1 × 109 spins and atomistic Monte Carlo simulations with up to 2 × 109 spins are possible, with near-ideal performance scaling.

H-Analysis and data-parallel physics-informed neural networks

We explore the data-parallel acceleration of physics-informed machine learning (PIML) schemes, with a focus on physics-informed neural networks (PINNs) for multiple graphics processing units (GPUs) architectures. In order to develop scale-robust and high-throughput PIML models for sophisticated applications which may require a large number of training points (e.g., involving complex and high-dimensional domains, non-linear operators or multi-physics), we detail a novel protocol based on h-analysis and data-parallel acceleration through the Horovod training framework. The protocol is backed by new convergence bounds for the generalization error and the train-test gap. We show that the acceleration is straightforward to implement, does not compromise training, and proves to be highly efficient and controllable, paving the way towards generic scale-robust PIML. Extensive numerical experiments with increasing complexity illustrate its robustness and consistency, offering a wide range of possibilities for real-world simulations.

Vectorized Matrix Formulation for Sound Power Determination Through Inverse Problems and Regularization Techniques

Sound power estimation is a widely investigated research topic. Some of its state-of-the-art measurement solutions include taking velocity values on the source boundary, discrete pressure assessment on a virtual outer surface or evaluations with a sound intensity probe. The increased availability of microphone arrays proposes a new tool to obtain these results. The inverse boundary element method, the equivalent source method, the Helmholtz equation least-squares, and an approach based on the radiation resistance matrix provide vectorized formulations for estimating sound power. In this study, the results for different regularization techniques on the sound power deviation from a discrete line source are presented based on a backward reconstruction scheme from simulated sound pressure field points on four microphone arrays. Additionally, in this study, time improvement between single-frequency calculations and multi-frequency execution on multiple graphics processing units is presented. The outcome shows good accuracy and reduced computation time in graphics card implementations.

The implementation of the three-dimensional unified gas-kinetic wave-particle method on multiple graphics processing units

To further improve the efficiency of the unified gas-kinetic wave-particle (UGKWP) method in hypersonic rarefied non-equilibrium flows, particularly the particle simulation process, we presented the first application of the three-dimensional UGKWP method to multiple graphics processing unit (GPU) devices in this study. The wave and particle evolution components of the method are addressed using cell and particle paralleling strategies, respectively, enabling the primary loop of the GPU-based UPKWP (GPU-UGKWP) to be executed entirely by the compute unified device architecture threads on GPU devices. Concurrently, communication issues between central processing unit (CPU) nodes are resolved by employing the message passing interface model. Additionally, we introduce a tailored memory management scheme for the GPU-UGKWP method, facilitating efficient access to the particle array. Performance comparisons reveal that, relative to a single Intel Xeon Gold 6148 CPU core, the Nvidia Tesla P100 achieves a total speedup of 34 using one GPU device, and 226 with eight GPU devices, and a single Nvidia Titan V GPU device attains a speedup of 62. The speedup outcomes on multiple CPU cores and GPU devices demonstrate that the GPU-based algorithm is better suited for computationally demanding tasks, particularly in particle-dominated simulations. As evidenced by the reduced calculation time for a hypersonic technology vehicle simulation performed on the P100 cluster, GPU devices significantly outperform their CPU counterparts.

A Multi-GPU Aggregation-Based AMG Preconditioner for Iterative Linear Solvers

We present and release in open source format a sparse linear solver which efficiently exploits heterogeneous parallel computers. The solver can be easily integrated into scientific applications that need to solve large and sparse linear systems on modern parallel computers made of hybrid nodes hosting Nvidia Graphics Processing Unit (GPU) accelerators. The work extends previous efforts of some of the authors in the exploitation of a single GPU accelerator and proposes an implementation, based on the hybrid MPI-CUDA software environment, of a Krylov-type linear solver relying on an efficient Algebraic MultiGrid (AMG) preconditioner already available in the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BootCMatchG</i> library. Our design for the hybrid implementation has been driven by the best practices for minimizing data communication overhead when multiple GPUs are employed, yet preserving the efficiency of the GPU kernels. Strong and weak scalability results of the new version of the library on well-known benchmark test cases are discussed. Comparisons with the Nvidia AmgX solution show a speedup, in the solve phase, up to 2.0x.

Adopting GPU computing to support DL-based Earth science applications

ABSTRACT With the advancement of Artificial Intelligence (AI) technologies and accumulation of big Earth data, Deep Learning (DL) has become an important method to discover patterns and understand Earth science processes in the past several years. While successful in many Earth science areas, AI/DL applications are often challenging for computing devices. In recent years, Graphics Processing Unit (GPU) devices have been leveraged to speed up AI/DL applications, yet computational performance still poses a major barrier for DL-based Earth science applications. To address these computational challenges, we selected five existing sample Earth science AI applications, revised the DL-based models/algorithms, and tested the performance of multiple GPU computing platforms to support the applications. Application software packages, performance comparisons across different platforms, along with other results, are summarized. This article can help understand how various AI/ML Earth science applications can be supported by GPU computing and help researchers in the Earth science domain better adopt GPU computing (such as supermicro, GPU clusters, and cloud computing-based) for their AI/ML applications, and to optimize their science applications to better leverage the computing device.

Accelerating 4D image reconstruction for magnetic resonance-guided radiotherapy.

Physiological motion impacts the dose delivered to tumours and vital organs in external beam radiotherapy and particularly in particle therapy. The excellent soft-tissue demarcation of 4D magnetic resonance imaging (4D-MRI) could inform on intra-fractional motion, but long image reconstruction times hinder its use in online treatment adaptation. Here we employ techniques from high-performance computing to reduce 4D-MRI reconstruction times below two minutes to facilitate their use in MR-guided radiotherapy. Four patients with pancreatic adenocarcinoma were scanned with a radial stack-of-stars gradient echo sequence on a 1.5T MR-Linac. Fast parallelised open-source implementations of the extra-dimensional golden-angle radial sparse parallel algorithm were developed for central processing unit (CPU) and graphics processing unit (GPU) architectures. We assessed the impact of architecture, oversampling and respiratory binning strategy on 4D-MRI reconstruction time and compared images using the structural similarity (SSIM) index against a MATLAB reference implementation. Scaling and bottlenecks for the different architectures were studied using multi-GPU systems. All reconstructed 4D-MRI were identical to the reference implementation (SSIM 0.99). Images reconstructed with overlapping respiratory bins were sharper at the cost of longer reconstruction times. The CPU +GPU implementation was over 17 times faster than the reference implementation, reconstructing images in 60 1 s and hyper-scaled using multiple GPUs. Respiratory-resolved 4D-MRI reconstruction times can be reduced using high-performance computing methods for online workflows in MR-guided radiotherapy with potential applications in particle therapy.

Phase-field lattice Boltzmann simulation of three-dimensional settling dendrite with natural convection during nonisothermal solidification of binary alloy

Understanding the motion and growth behaviors of equiaxed dendrites during solidification is important for predicting macrosegregation. In this study, we develop a phase-field lattice Boltzmann (PF-LB) simulation method for the settling and growth of an equiaxed dendrite during the nonisothermal solidification of a binary alloy. The PF-LB computations are accelerated by employing parallel computation using multiple graphic processing units (GPUs) and the octree block-structured adaptive mesh refinement method, which incorporates multiple mesh and time increment methods. By using the developed method, we can simulate the three-dimensional long-distance settling dendrite while considering the effects of latent heat release and natural convection. From the simulation results, we confirm that the natural convection due to the high solute concentration around a dendrite reduces the settling velocity. In addition, we observe that the temperature increase owing to latent heat release slows dendrite growth, which in turn slightly slows the settling velocity. From these results, we confirm that the effects of latent heat release and natural convection are not negligible in the quantitative evaluation of settling dendrites.

RTGPU: Real-Time GPU Scheduling of Hard Deadline Parallel Tasks With Fine-Grain Utilization

Many emerging cyber-physical systems, such as autonomous vehicles and robots, rely heavily on artificial intelligence and machine learning algorithms to perform important system operations. Since these highly parallel applications are computationally intensive, they need to be accelerated by graphics processing units (GPUs) to meet stringent timing constraints. However, despite the wide adoption of GPUs, efficiently scheduling multiple GPU applications while providing rigorous real-time guarantees remains a challenge. Each GPU application has multiple CPU execution and memory copy segments, with GPU kernels running on different hardware resources. Because of the complicated interactions between heterogeneous segments of parallel tasks, high schedulability is hard to achieve with conventional approaches. This paper proposes RTGPU, which combines fine-grain GPU partitioning on the system side with a novel scheduling algorithm on the theory side. Through system and theory co-design, RTGPU achieves superior system throughput and real-time schedulability. In this paper, we start by building a model for the CPU and memory copy segments. Leveraging persistent threads, we then implement fine-grained GPU partitioning with improved performance through interleaved execution. To reap the benefits of fine-grained GPU partitioning and schedule multiple parallel GPU applications, we propose a novel real-time scheduling algorithm based on federated scheduling and grid search with uniprocessor fixed-priority scheduling. Our approach provides real-time guarantees to meet hard deadlines, and achieves over 11% improvement in system throughput and up to 57% schedulability improvement compared with previous work. We validate and evaluate RTGPU on NVIDIA GTX1080Ti GPU systems. Our system side techniques can be applied on mainstream NVIDIA GPUs, and the proposed scheduling theory can be used in general heterogeneous computing platforms which have a similar task execution pattern.

Multi-Objective Neural Evolutionary Algorithm for Combinatorial Optimization Problems.

There has been a recent surge of success in optimizing deep reinforcement learning (DRL) models with neural evolutionary algorithms. This type of method is inspired by biological evolution and uses different genetic operations to evolve neural networks. Previous neural evolutionary algorithms mainly focused on single-objective optimization problems (SOPs). In this article, we present an end-to-end multi-objective neural evolutionary algorithm based on decomposition and dominance (MONEADD) for combinatorial optimization problems. The proposed MONEADD is an end-to-end algorithm that utilizes genetic operations and rewards signals to evolve neural networks for different combinatorial optimization problems without further engineering. To accelerate convergence, a set of nondominated neural networks is maintained based on the notion of dominance and decomposition in each generation. In inference time, the trained model can be directly utilized to solve similar problems efficiently, while the conventional heuristic methods need to learn from scratch for every given test problem. To further enhance the model performance in inference time, three multi-objective search strategies are introduced in this work. Our experimental results clearly show that the proposed MONEADD has a competitive and robust performance on a bi-objective of the classic travel salesman problem (TSP), as well as Knapsack problem up to 200 instances. We also empirically show that the designed MONEADD has good scalability when distributed on multiple graphics processing units (GPUs).

Space-to-speed architecture supporting acceleration on VHR image processing

One of the major focuses in the remote sensing community is the rapid processing of deep neural networks (DNNs) on very high resolution (VHR) aerial images. Few studies have investigated the acceleration of training and prediction by optimizing the architecture of the DNN system rather than designing a lightweight DNN. Parallel processing using multiple graphics processing units (GPUs) increases VHR image processing performance. It drives extremely large and frequent data transfers (input/output(I/O)) from random access memory (RAM) to GPU memory. As a result, the system bus congestion causes the system to hang, resulting in long latency in training/predicting. In this paper, we evaluate the causes of long latency and propose a space-to-speed (S2S) DNN system to overcome the aforementioned challenges. A three-level memory system aiming to reduce data transfer during system operation is presented. Distribution optimization with parallel processing was used to accelerate the training. Training optimizations on VHR images (such as hot-zone searching and image/ground truth queues for data saving) were used to train the VHR images efficiently. Inference optimization was performed to speed up prediction in the release mode. To verify the efficiency of the proposed system, we used aerial image labeling from the Institut National de Recherche en Informatique et en Automatique (INRIA) and benchmarks from the Massachusetts Institute of Technology Aerial Imagery for Roof Segmentation (MITAIRS) to test the system performance and accuracy. Without the loss of accuracy, the S2S system improved prediction speed on the testing dataset by eight GPUs in a normal setting in both the INRIA dataset (from 534 to 72 s) and the MITAIRS dataset (818 to 120 s). With the prediction in half-float (using float-16 data), an 8-GPU parallel processing increased the speed to 38 s in the INRIA dataset and 83 s in the MITAIRS dataset. In a pressure test, our proposed system operated on 18,000 images with a size of 5000 × 5000 from 18.2 to 1.8 h with the prediction in full-float (using float-32 data) and 43 min with the prediction in half-float, increasing the speed by a factor of 9.78 and 25.3, respectively, when compared to system runs without optimization.

Accelerating the density-functional tight-binding method using graphical processing units.

Acceleration of the density-functional tight-binding (DFTB) method on single and multiple graphical processing units (GPUs) was accomplished using the MAGMA linear algebra library. Two major computational bottlenecks of DFTB ground-state calculations were addressed in our implementation: the Hamiltonian matrix diagonalization and the density matrix construction. The code was implemented and benchmarked on two different computer systems: (1) the SUMMIT IBM Power9 supercomputer at the Oak Ridge National Laboratory Leadership Computing Facility with 1-6 NVIDIA Volta V100 GPUs per computer node and (2) an in-house Intel Xeon computer with 1-2 NVIDIA Tesla P100 GPUs. The performance and parallel scalability were measured for three molecular models of 1-, 2-, and 3-dimensional chemical systems, represented by carbon nanotubes, covalent organic frameworks, and water clusters.