Single Graphics Processing Units Research Articles

Digital rock physics: Calculation of effective elastic properties of heterogeneous materials using graphical processing units (GPUs)

An application based on graphical processing units (GPUs) applied to 3-D digital images is described for computing the linear anisotropic elastic properties of heterogeneous materials. The application can also retrieve the property contribution tensors of individual inclusions of any shape. The code can be executed on professional GPUs as well as on a basic laptop or personal computer Nvidia GPUs. The application is extremely fast: a calculation of the effective elastic properties of volumes consisting of about 7 million voxel elements (1913) takes less than 4 s of computational time using a single A100 GPU; 3 min for 100 million voxel elements (4793) using a single A100 GPU; 14 min for 350 million voxel elements (7033) using a single A100 GPU. Several comparisons against analytical solutions are provided. In addition, an evaluation of the anisotropic effective elastic properties of a 3-D digital image of a cracked Carrara marble sample is presented. The software can be downloaded from a permanent repository Zenodo, the link with a doi is given in the manuscript.

Large scale multi-GPU based parallel traffic simulation for accelerated traffic assignment and propagation

Traffic simulation is a critical tool for congestion analysis, travel time estimation, and route optimization in urban planning, benefiting navigation apps, transportation network companies, and state agencies. Traditionally, traffic micro-simulation frameworks are based on road segments and can only support a limited number of main roads. Efficient traffic simulation on a regional scale remains a significant challenge due to the complexity of urban mobility and the large scale of spatiotemporal data. This paper introduces a Large Scale Multi-GPU Parallel Computing based Regional Scale Traffic Simulation Framework (LPSim), which leverages graphical processing unit (GPU) parallel computing to address these challenges. LPSim utilizes a multi-GPU architecture to simulate extensive and dynamic traffic networks with high fidelity and reduced computation time. Using the parallel processing capabilities of GPUs, LPSim can perform tens of millions of individual vehicle dynamics simulations simultaneously, significantly outperforming traditional CPU-based approaches. The framework is designed to be scalable and can easily accommodate the increasing complexity of traffic simulations. We present the theory behind GPU-based traffic simulation, the architecture of single- and multi-GPU based simulations, and the graph partition strategies that enhance computation resource load balance. Our experimental results demonstrate the effectiveness of LPSim in simulating large-scale traffic scenarios. LPSim is capable of completing simulations of 2.82 million trips in just 6.28 min on a single GPU machine equipped with 5120 CUDA cores (Tesla V100-SXM2). Furthermore, utilizing a Google Cloud instance with two NVIDIA V100 GPUs, which collectively offer 10240 CUDA cores, LPSim successfully simulates 9.01 million trips within 21.16 min. We further tested our simulator with the same demand on dual NVIDIA A100-PCIE-40GB GPUs, which finished the simulation in 0.0398 h, approximately 113 times faster than the same simulation scenario running on an Intel(R) Xeon(R) Gold 6326 CPU @ 2.90 GHz, which takes 4.49 h to complete. This performance not only demonstrates its speed and scalability advantages over traditional simulation techniques but also highlights LPSim’s unique position as the first traffic simulation framework that is scalable for both single- and multiple-GPU configurations. Consequently, LPSim provides an invaluable tool for individuals and extensive research teams alike, enabling the acquisition of large-scale traffic simulation results in a time-efficient manner. LPSim code is available at: https://github.com/Xuan-1998/LPSim

High-speed turbulent flows towards the exascale: STREAmS-2 porting and performance

Exascale High Performance Computing (HPC) represents a tremendous opportunity to push the boundaries of Computational Fluid Dynamics (CFD), but despite the consolidated trend towards the use of Graphics Processing Units (GPUs), programmability is still an issue. STREAmS-2 (Bernardini et al. Comput. Phys. Commun. 285 (2023) 108644) is a compressible solver for canonical wall-bounded turbulent flows capable of harvesting the potential of NVIDIA GPUs. Here we extend the already available CUDA Fortran backend with a novel HIP backend targeting AMD GPU architectures. The main implementation strategies are discussed along with a novel Python tool that can generate the HIP and CPU code versions allowing developers to focus their attention only on the CUDA Fortran backend. Single GPU performance is analysed focusing on NVIDIA A100 and AMD MI250x cards which are currently at the core of several HPC clusters. The gap between peak GPU performance and STREAmS-2 performance is found to be generally smaller for NVIDIA cards. Roofline analysis allows tracing this behavior to unexpectedly different computational intensities of the same kernel using the two cards. Additional single-GPU comparisons are performed to assess the impact of grid size, number of parallelized loops, thread masking and thread divergence. Parallel performance is measured on the two largest EuroHPC pre-exascale systems, LUMI (AMD GPUs) and Leonardo (NVIDIA GPUs). Strong scalability reveals more than 80% efficiency up to 16 nodes for Leonardo and up to 32 for LUMI. Weak scalability shows an impressive efficiency of over 95% up to the maximum number of nodes tested (256 for LUMI and 512 for Leonardo). This analysis shows that STREAmS-2 is the perfect candidate to fully exploit the power of current pre-exascale HPC systems in Europe, allowing users to simulate flows with over a trillion mesh points, thus reducing the gap between the Reynolds numbers achievable in high-fidelity simulations and those of real engineering applications.

Enhanced seagull optimization for enhanced accuracy in CUDA-accelerated Levenberg–Marquardt backpropagation neural networks for earthquake forecasting

Hyperparameter tuning is crucial for enhancing the accuracy and reliability of artificial neural networks (ANNs). This study presents an optimization of the Levenberg–Marquardt backpropagation neural network (LM-BPNN) by integrating an improved seagull optimization algorithm (ISOA). The proposed ISOA-LM-BPNN model is designed to forecast earthquakes in the Caribbean region. The study further explores the impact of data and model parallelism, revealing that hybrid parallelism effectively mitigates the limitations of both. This leads to substantial gains in throughput and overall performance. To address computational demands, this model leverages the compute unified device architecture (CUDA) framework, enabling hybrid parallelism on graphics processing units (GPUs). This approach significantly enhances the model’s computational speed. The experimental results demonstrate that the ISOA-LM-BPNN model achieves a 20% improvement in accuracy compared to four baseline algorithms across three diverse datasets. The integration of ISOA with LM-BPNN refines the neural network’s hyperparameters, leading to more precise earthquake predictions. Additionally, the model’s computational efficiency is evidenced by a 56% speed increase when utilizing a single GPU, and an even greater acceleration with dual GPUs connected via NVLink compared to traditional CPU-based computations. The findings underscore the potential of ISOA-LM-BPNN as a robust tool for earthquake forecasting, combining high accuracy with enhanced computational speed, making it suitable for real-time applications in seismic monitoring and early warning systems.

Multi-GPU RI-HF Energies and Analytic Gradients─Toward High-Throughput Ab Initio Molecular Dynamics.

This article presents an optimized algorithm and implementation for calculating resolution-of-the-identity Hartree-Fock (RI-HF) energies and analytic gradients using multiple graphics processing units (GPUs). The algorithm is especially designed for high throughput ab initio molecular dynamics simulations of small and medium size molecules (10-100 atoms). Key innovations of this work include the exploitation of multi-GPU parallelism and a workload balancing scheme that efficiently distributes computational tasks among GPUs. Our implementation also employs techniques for symmetry utilization, integral screening, and leveraging sparsity to optimize memory usage. Computational results show that the implementation achieves significant performance improvements, including over 3 × speedups in single GPU AIMD throughput compared to previous GPU-accelerated RI-HF and traditional HF methods. Furthermore, utilizing multiple GPUs can provide superlinear speedup when the additional aggregate GPU memory allows for the storage of decompressed three-center integrals.

STORMM: Structure and topology replica molecular mechanics for chemical simulations.

The Structure and TOpology Replica Molecular Mechanics (STORMM) code is a next-generation molecular simulation engine and associated libraries optimized for performance on fast, vectorized central processor units and graphics processing units (GPUs) with independent memory and tens of thousands of threads. STORMM is built to run thousands of independent molecular mechanical calculations on a single GPU with novel implementations that tune numerical precision, mathematical operations, and scarce on-chip memory resources to optimize throughput. The libraries are built around accessible classes with detailed documentation, supporting fine-grained parallelism and algorithm development as well as copying or swapping groups of systems on and off of the GPU. A primary intention of the STORMM libraries is to provide developers of atomic simulation methods with access to a high-performance molecular mechanics engine with extensive facilities to prototype and develop bespoke tools aimed toward drug discovery applications. In its present state, STORMM delivers molecular dynamics simulations of small molecules and small proteins in implicit solvent with tens to hundreds of times the throughput of conventional codes. The engineering paradigm transforms two of the most memory bandwidth-intensive aspects of condensed-phase dynamics, particle-mesh mapping, and valence interactions, into compute-bound problems for several times the scalability of existing programs. Numerical methods for compressing and streamlining the information present in stored coordinates and lookup tables are also presented, delivering improved accuracy over methods implemented in other molecular dynamics engines. The open-source code is released under the MIT license.

O-260 Identifying seminiferous tubules for sperm retrieval during microdissection testicular sperm extraction (M-TESE) using deep learning

Abstract Study question Is it possible to utilize artificial intelligence (AI) in the selection of seminiferous tubules for sperm retrieval during microdissection testicular sperm extraction (M-TESE)? Summary answer AI algorithm was able to segment seminiferous tubules deemed to be good candidates for sperm retrieval with IoU of 0.746 and F1 score of 72.5%. What is known already M-TESE is an advanced surgical procedure reserved for sperm retrieval in men with non-obstructive azoospermia (NOA). Meta-analysis have shown M-TESE might be superior in relation to sperm retrieval rates and postsurgical complications when compared to conventional techniques. Limitations of the technique are the relatively long learning curve for the surgeons and the longer intraoperative time required to identify the tubules with active spermatogenesis. Convolutional neural networks (CNNs) have been shown to successfully identify areas of interest in medical images such as CT and MRI scans. The possibility of utilizing trained CNNs in M-TESE procedures has so far not been studied. Study design, size, duration This is a cross sectional study analyzing image data gathered during M-TESE procedures including the first five men enrolled in a prospective study designed specifically to test the utilization of AI in the treatment of NOA. All participants signed written informed consent. Images were collected intraoperatively with low and high resolution cameras according to predefined protocol. Participants/materials, setting, methods CNN model with down and upsampling architecture was trained using patch-based technique with corresponding labelled truth image pairs based on combination of semi-automated hue saturation value thresholding and manual segmentation of seminiferous tubules. Tubules were allocated into two different classes by the operating surgeon according to the visual perception of their morphology as type 1 (i.e. larger, opaque, good candidates for micro dissection) and type 2 (i.e. sclerotic or atrophic; unlikely to harbor active spermatogenesis). Main results and the role of chance CNN models were trained on 1809 patches with corresponding ground truth segmented images. Two algorithms were tested: the first was trained solely to detect and semantically distinguish type 1 seminiferous tubules. The second was trained to semantically segment the major anatomical structures: tunica albuginea, testicular parenchyma, blood vessels; lobules and the two types of seminiferous tubules describe above. Validation was performed on 988 image patches not previously shown to the algorithm. After grid search for optimal learning rate and augmentation the first algorithm was able to segment type 1 tubules with F1 score of 72.5% and Jaccard Index (IoU) of 0.746. Training took 856 minutes on 48 core high performance computer (HPC) equipped with single graphics processing unit (GPU). Same computational environment was used in the training of the second algorithm. This model was able to segment anatomical structures such as parenchyma (IoU=0.732) and testicular lobules (IoU=0.683) but performed sub optimally on the clinically relevant type 1 tubules (IoU=0.432). The first trained algorithm was able to segment a single high resolution surgical image in 3.4 seconds on an HPC. However, the same procedure took 5.5 minutes on standard desktop computer with 8 core Intel processor without GPU. Limitations, reasons for caution The number of patients is relatively small and therefore more training data would be needed to achieve robust performance. Training data were labelled by single surgeon limiting the generalizability and possibly biasing the selection process. The need of HPC environment for real time segmentation can limit the clinical use. Wider implications of the findings This pilot study shows the potential of AI to help in choosing suitable tubules for micro dissection, shortening the learning curve for young andrological surgeons. The access to cluster based computation is getting easier by the day and can soon allow instant intraoperative AI segmentation, thus shortening the surgical procedure. Trial registration number NA

Do we really need machine learning interatomic potentials for modeling amorphous metal oxides? Case study on amorphous alumina by recycling an existing ab initio database

In this study, we critically evaluate the performance of various interatomic potentials/force fields against a benchmark ab initio database for bulk amorphous alumina. The interatomic potentials tested in this work include all major fixed charge and variable charge models developed to date for alumina. Additionally, we introduce a novel machine learning interatomic potential constructed using the NequIP framework based on graph neural networks. Our findings reveal that the fixed-charge potential developed by Matsui and coworkers offers the most optimal balance between computational efficiency and agreement with ab initio data for stoichiometric alumina. Such balance cannot be provided by machine learning potentials when comparing performance with Matsui potential on the same computing infrastructure using a single Graphical Processing Unit. For non-stoichiometric alumina, the variable charge potentials, in particular ReaxFF, exhibit an impressive concordance with density functional theory calculations. However, our NequIP potentials trained on a small fraction of the ab initio database easily surpass ReaxFF in terms of both accuracy and computational performance. This is achieved without large overhead in terms of potential fitting and fine-tuning, often associated with the classical potential development process as well as training of standard deep neural network potentials, thus advocating for the use of data-efficient machine learning potentials like NequIP for complex cases of non-stoichiometric amorphous oxides.

A GPU-accelerated computational fluid dynamics solver for assessing shear-driven indoor airflow and virus transmission by scale-resolved simulations

We explore the applicability of MATLAB for 3D computational fluid dynamics (CFD) of shear-driven indoor airflows. A new scale-resolving, large-eddy simulation (LES) solver titled DNSLABIB is proposed for MATLAB utilizing graphics processing units (GPUs). In DNSLABIB, the finite difference method is applied for the convection and diffusion terms while a Poisson equation solver based on the fast Fourier transform (FFT) is employed for the pressure. The immersed boundary method (IBM) for Cartesian grids is proposed to model solid walls and objects, doorways, and air ducts by binary masking of the solid/fluid domains. The solver is validated in two canonical reference cases and against experimental data. Then, we demonstrate the validity of DNSLABIB in a room geometry by comparing the results against another CFD software (OpenFOAM). Next, we demonstrate the solver performance in several isothermal indoor ventilation configurations and the implications of the results are discussed in the context of airborne transmission of COVID-19. The novel numerical findings using the new CFD solver are as follows. First, a linear scaling of DNSLABIB is demonstrated and a speed-up by a factor of 3-4 is also noted in comparison to similar OpenFOAM simulations. Second, ventilation in three different indoor geometries are studied at both low (0.1 m/s) and high (1 m/s) airflow rates corresponding to Re=5000 and Re=50000. An analysis of the indoor CO2 concentration is carried out as the room is emptied from stale, high CO2 content air. We estimate the air changes per hour (ACH) values for three different room geometries and show that the numerical estimates from 3D CFD simulations differ by 80%–150% (Re=50000) and 75%–140% (Re=5000) from the theoretical ACH value based on the perfect mixing assumption. Third, the analysis of the CO2 probability distributions (PDFs) indicates a relatively non-uniform distribution of fresh air indoors. Fourth, utilizing a time-dependent Wells-Riley analysis, an example is provided on the growth of the cumulative infection risk which is shown to reduce rapidly after the ventilation is started. The average infection risk is shown to reduce by a factor of 2 for lower ventilation rates (ACH=3.4-6.3) and 10 for the higher ventilation rates (ACH=37-64). Finally, we utilize the new solver to comment on respiratory particle transport indoors. A key contribution of the paper is to provide an efficient, GPU compatible CFD solver environment enabling scale-resolved simulations (LES/DNS) of airflow in large indoor geometries on a single GPU designed for high-performance computing. The demonstrated efficacy of MATLAB for GPU computing indicates a high potential of DNSLABIB for various future developments on airflow prediction.

Iterative Reconstruction of Micro Computed Tomography Scans Using Multiple Heterogeneous GPUs.

Graphics processing units (GPUs) facilitate massive parallelism and high-capacity storage, and thus are suitable for the iterative reconstruction of ultrahigh-resolution micro computed tomography (CT) scans by on-the-fly system matrix (OTFSM) calculation using ordered subsets expectation maximization (OSEM). We propose a finite state automaton (FSA) method that facilitates iterative reconstruction using a heterogeneous multi-GPU platform through parallelizing the matrix calculations derived from a ray tracing system of ordered subsets. The FSAs perform flow control for parallel threading of the heterogeneous GPUs, which minimizes the latency of launching ordered-subsets tasks, reduces the data transfer between the main system memory and local GPU memory, and solves the memory-bound of a single GPU. In the experiments, we compared the operation efficiency of OS-MLTR for three reconstruction environments. The heterogeneous multiple GPUs with job queues for high throughput calculation speed is up to five times faster than the single GPU environment, and that speed up is nine times faster than the heterogeneous multiple GPUs with the FIFO queues of the device scheduling control. Eventually, we proposed an event-triggered FSA method for iterative reconstruction using multiple heterogeneous GPUs that solves the memory-bound issue of a single GPU at ultrahigh resolutions, and the routines of the proposed method were successfully executed on each GPU simultaneously.

High-order thread-safe lattice Boltzmann model for high performance computing turbulent flow simulations

We present a highly optimized thread-safe lattice Boltzmann model in which the non-equilibrium part of the distribution function is locally reconstructed via recursivity of Hermite polynomials. Such a procedure allows the explicit incorporation of non-equilibrium moments of the distribution up to the order supported by the lattice. Thus, the proposed approach increases accuracy and stability at low viscosities without compromising performance and amenability to parallelization with respect to standard lattice Boltzmann models. The high-order thread-safe lattice Boltzmann is tested on two types of turbulent flows, namely, the turbulent channel flow at Reτ=180 and the axisymmetric turbulent jet at Re = 7000; it delivers results in excellent agreement with reference data [direct numerical simulations (DNS), theory, and experiments] and (a) achieves peak performance [∼5×1012 floating point operations (FLOP) per second and an arithmetic intensity of ∼7 FLOP/byte on a single graphic processing unit] by significantly reducing the memory footprint, (b) retains the algorithmic simplicity of standard lattice Boltzmann computing, and (c) allows to perform stable simulations at vanishingly low viscosities. Our findings open attractive prospects for high-performance simulations of realistic turbulent flows on GPU-based architectures. Such expectations are confirmed by excellent agreement among lattice Boltzmann, experimental, and DNS reference data.

Multi-Device Parallel MRI Reconstruction: Efficient Partitioning for Undersampled 5D Cardiac CINE.

Cardiac CINE, a form of dynamic cardiac MRI, is indispensable in the diagnosis and treatment of heart conditions, offering detailed visualization essential for the early detection of cardiac diseases. As the demand for higher-resolution images increases, so does the volume of data requiring processing, presenting significant computational challenges that can impede the efficiency of diagnostic imaging. Our research presents an approach that takes advantage of the computational power of multiple Graphics Processing Units (GPUs) to address these challenges. GPUs are devices capable of performing large volumes of computations in a short period, and have significantly improved the cardiac MRI reconstruction process, allowing images to be produced faster. The innovation of our work resides in utilizing a multi-device system capable of processing the substantial data volumes demanded by high-resolution, five-dimensional cardiac MRI. This system surpasses the memory capacity limitations of single GPUs by partitioning large datasets into smaller, manageable segments for parallel processing, thereby preserving image integrity and accelerating reconstruction times. Utilizing OpenCL technology, our system offers adaptability and cross-platform functionality, ensuring wider applicability. The proposed multi-device approach offers an advancement in medical imaging, accelerating the reconstruction process and facilitating faster and more effective cardiac health assessment.

Graphics-processing-unit-accelerated ice flow solver for unstructured meshes using the Shallow-Shelf Approximation (FastIceFlo v1.0.1)

Abstract. Ice-sheet flow models capable of accurately projecting their future mass balance constitute tools to improve flood risk assessment and assist sea-level rise mitigation associated with enhanced ice discharge. Some processes that need to be captured, such as grounding-line migration, require high spatial resolution (under the kilometer scale). Conventional ice flow models mainly execute on central processing units (CPUs), which feature limited parallel processing capabilities and peak memory bandwidth. This may hinder model scalability and result in long run times, requiring significant computational resources. As an alternative, graphics processing units (GPUs) are ideally suited for high spatial resolution, as the calculations can be performed concurrently by thousands of threads, processing most of the computational domain simultaneously. In this study, we combine a GPU-based approach with the pseudo-transient (PT) method, an accelerated iterative and matrix-free solution strategy, and investigate its performance for finite elements and unstructured meshes with application to two-dimensional (2-D) models of real glaciers at a regional scale. For both the Jakobshavn and Pine Island glacier models, the number of nonlinear PT iterations required to converge a given number of vertices (N) scales in the order of 𝒪(N1.2) or better. We further compare the performance of the PT CUDA C implementation with a standard finite-element CPU-based implementation using the price-to-performance metric. The price of a single Tesla V100 GPU is 1.5 times that of two Intel Xeon Gold 6140 CPUs. We expect a minimum speedup of at least 1.5 times to justify the Tesla V100 GPU price to performance. Our developments result in a GPU-based implementation that achieves this goal with a speedup beyond 1.5 times. This study represents a first step toward leveraging GPU processing power, enabling more accurate polar ice discharge predictions. The insights gained will benefit efforts to diminish spatial resolution constraints at higher computing performance. The higher computing performance will allow for ensembles of ice-sheet flow simulations to be run at the continental scale and higher resolution, a previously challenging task. The advances will further enable the quantification of model sensitivity to changes in upcoming climate forcings. These findings will significantly benefit process-oriented sea-level-projection studies over the coming decades.

PARALLEL AND DISTRIBUTED COMPUTING TECHNOLOGIES FOR AUTONOMOUS VEHICLE NAVIGATION

Context. Autonomous vehicles are becoming increasingly popular, and one of the important modern challenges in their development is ensuring their effective navigation in space and movement within designated lanes. This paper examines a method of spatial orientation for vehicles using computer vision and artificial neural networks. The research focused on the navigation system of an autonomous vehicle, which incorporates the use of modern distributed and parallel computing technologies.
 Objective. The aim of this work is to enhance modern autonomous vehicle navigation algorithms through parallel training of artificial neural networks and to determine the optimal combination of technologies and nodes of devices to increase speed and enable real-time decision-making capabilities in spatial navigation for autonomous vehicles.
 Method. The research establishes that the utilization of computer vision and neural networks for road lane segmentation proves to be an effective method for spatial orientation of autonomous vehicles. For multi-core computing systems, the application of parallel programming technology, OpenMP, for neural network training on processors with varying numbers of parallel threads increases the algorithm’s execution speed. However, the use of CUDA technology for neural network training on a graphics processing unit significantly enhances prediction speeds compared to OpenMP. Additionally, the feasibility of employing PyTorch Distributed Data Parallel (DDP) technology for training the neural network across multiple graphics processing units (nodes) simultaneously was explored. This approach further improved prediction execution times compared to using a single graphics processing unit.
 Results. An algorithm for training and prediction of an artificial neural network was developed using two independent nodes, each equipped with separate graphics processing units, and their synchronization for exchanging training results after each epoch, employing PyTorch Distributed Data Parallel (DDP) technology. This approach allows for scalable computations across a higher number of resources, significantly expediting the model training process.
 Conclusions. The conducted experiments have affirmed the effectiveness of the proposed algorithm, warranting the recommendation of this research for further advancement in autonomous vehicles and enhancement of their navigational capabilities. Notably, the research outcomes can find applications in various domains, encompassing automotive manufacturing, logistics, and urban transportation infrastructure. The obtained results are expected to assist future researchers in understanding the most efficient hardware and software resources to employ for implementing AI-based navigation systems in autonomous vehicles. Prospects for future investigations may encompass refining the accuracy of the proposed parallel algorithm without compromising its efficiency metrics. Furthermore, there is potential for experimental exploration of the proposed algorithm in more intricate practical scenarios of diverse nature and dimensions.

A single-phase GPU-accelerated surface tension model using SPH

This paper presents an accelerated single-phase surface tension smoothed particle hydrodynamics (SPH) solver developed to run entirely on a graphics processing unit (GPU) capable of simulating millions of particles in three dimensions on a single GPU. The single-phase surface tension model is augmented with a contact line force to improve the prediction of the physics at the liquid-solid contact point. The surface tension model uses the modified dynamic boundary condition (mDBC) to impose no-slip conditions at the wall boundary. To enable simulations with millions of particles, the single-phase surface tension model has been implemented in the open-source SPH code DualSPHysics to exploit the GPU acceleration, paying special attention to the size of the kernel support and integration of the neighbour lists. The new scheme is validated using 2-D and 3-D test cases including drop deformation, drop oscillation, Rayleigh-Plateau instability and surface contact angles. The results show a good agreement with the analytical solutions with a standard spatial convergence behaviour. Profiling the new surface tension solver shows an additional computational complexity. The performance analysis shows that the new code has a speed up of up two orders of magnitude (x70-80) compared to the CPU-only code. Profiling the new CUDA kernels shows they have the near identical performance metrics with the main CUDA kernels in the original DualSPHysics solver.

GPU acceleration of conjugate gradient method obtaining Green's function for transport-property calculation

We present a graphics processing unit (GPU) acceleration procedure for the transport-property calculation of the RSPACE code, which uses density functional theory and the real-space finite-difference method. Porting the RSPACE code to run on a GPU using Open Accelerator and libraries results in the improvement of the computational speed and efficiency compared with a central processing unit. Furthermore, we achieved a speed up of ∼ 4x compared with a single GPU and effective processing in a multi-GPU environment by parallelizing in a way that minimizes communication and synchronization. Our GPU-acceleration procedure is demonstrated by the calculation of electron-transport properties of (4,4) carbon nanotubes (CNTs) with a 5-8-5 defect for which the calculations are suitable for parallel processing on GPUs. It is found that the information loss is strongly affected by the defect geometry when electron waves propagating in CNTs are utilized as the information transfer signal.

Accelerating micromagnetic and atomistic simulations using multiple GPUs

It is shown that micromagnetic and atomistic spin dynamics simulations can use multiple graphical processing units (GPUs) not only to reduce computation time but also to allow for a larger simulation size than is possible on a single GPU. While interactions that depend on neighboring spins, such as exchange interactions, may be implemented efficiently by transferring data between GPUs using halo regions or direct memory accesses, implementing the long-range demagnetizing interaction is the main difficulty in achieving good performance scaling, where the data transfer rate between GPUs is a significant bottleneck. A multi-GPU convolution algorithm is developed here, which relies on single-GPU FFTs executed in parallel. It is shown that even for micromagnetic simulations where the demagnetizing interaction computation time dominates, good performance scaling may be achieved, with speedup factors up to 1.8, 2.5, and 3.1, for two, three, and four GPUs, respectively. The code developed here can be used for any number of GPUs in parallel, with performance scaling strongly dependent on the inter-GPU data transfer rate and connection topology. This is further improved in micromagnetic simulations, which include a spin transport solver, obtaining speedup factors up to 1.96, 2.8, and 3.7, for two, three, and four GPUs, respectively. The best case scenario is obtained for atomistic simulations, where the demagnetizing interaction is implemented with spin-averaged cells. Using a single workstation with four GPUs, it is shown that atomistic spin dynamics simulations with up to 1 × 109 spins and atomistic Monte Carlo simulations with up to 2 × 109 spins are possible, with near-ideal performance scaling.

Particle-resolved thermal lattice Boltzmann simulation using OpenACC on multi-GPUs

We utilize the Open Accelerator (OpenACC) approach for graphics processing unit (GPU) accelerated particle-resolved thermal lattice Boltzmann (LB) simulation. We adopt the momentum-exchange method to calculate fluid-particle interactions to preserve the simplicity of the LB method. To address load imbalance issues, we extend the indirect addressing method to collect fluid-particle link information at each timestep and store indices of fluid-particle link in a fixed index array. We simulate the sedimentation of 4,800 hot particles in cold fluids with a domain size of 40002, and the simulation achieves 1750 million lattice updates per second (MLUPS) on a single GPU. Furthermore, we implement a hybrid OpenACC and message passing interface (MPI) approach for multi-GPU accelerated simulation. This approach incorporates four optimization strategies, including building domain lists, utilizing request-answer communication, overlapping communications with computations, and executing computation tasks concurrently. By reducing data communication between GPUs, hiding communication latency through overlapping computation, and increasing the utilization of GPU resources, we achieve improved performance, reaching 10846 MLUPS using 8 GPUs. Our results demonstrate that the OpenACC-based GPU acceleration is promising for particle-resolved thermal lattice Boltzmann simulation.

A hybrid CPU/GPU method for Hartree-Fock self-consistent-field calculation.

The calculation of two-electron repulsion integrals (ERIs) is a crucial aspect of Hartree-Fock calculations. In computing the ERIs of varying angular momentum, both the central processing unit (CPU) and the graphics processing unit (GPU) have their respective advantages. To accelerate the ERI evaluation and Fock matrix generation, a hybrid CPU/GPU method has been proposed to maximize the computational power of both CPU and GPU while overlapping the CPU and GPU computations. This method employs a task queue where each task corresponds to ERIs with the same angular momentum. The queue begins with ERIs of low angular momentum, which are computationally efficient on GPUs, and ends with ERIs of high angular momentum, which are better suited for CPU computation. CPUs and GPUs dynamically grab and complete tasks from the start and end of the queue using OpenMP dynamic scheduling until all tasks are finished. The hybrid CPU/GPU computation offers the advantage of enabling calculations with arbitrary angular momentum. Test calculations showed that the hybrid CPU/GPU algorithm is more efficient than "GPU-only" when using a single GPU. However, as more GPUs are involved, the advantage diminishes or disappears. The scaling exponents of the hybrid method were slightly higher than "GPU-only," but the pre-exponent factor was significantly lower, making the hybrid method more effective overall.

IDEFIX: A versatile performance-portable Godunov code for astrophysical flows

Context. The exascale super-computers becoming available rely on hybrid energy-efficient architectures that involve an accelerator such as a graphics processing unit (GPU). Leveraging the computational power of these machines often means a significant rewrite of the numerical tools each time a new architecture becomes available. Aims. We present IDEFIX, a new code for astrophysical flows that relies on the KOKKOS meta-programming library to guarantee performance portability on a wide variety of architectures while keeping the code as simple as possible to the user. Methods. IDEFIX is based on a Godunov finite-volume method that solves the nonrelativistic hydrodynamical (HD) and magnetohy-drodynamical (MHD) equations on various grid geometries. IDEFIX includes a large choice of solvers and several additional modules (constrained transport, orbital advection, nonideal MHD), allowing users to address complex astrophysical problems. Results. IDEFIX has been successfully tested on Intel and AMD CPUs (up to 131 072 CPU cores on Irene-Rome at TGCC) as well as NVidia and AMD GPUs (up to 1024 GPUs on Adastra at CINES). IDEFIX achieves more than 108 cell s−1 in MHD on a single NVidia V100 GPU and 3 × 1011 cell s−1 on 256 Adastra nodes (1024 GPUs) with 95% parallelization efficiency (compared to single node). For the same problem, IDEFIX is up to six times more energy efficient on GPUs compared to Intel Cascade Lake CPUs. Conclusions. IDEFIX is now a mature exascale-ready open-source code that can be used on a large variety of astrophysical and fluid dynamics applications.