CUDA Fortran Research Articles

Typhoon storm surge ensemble forecast based on GPU technique

The accuracy of typhoon forecasts plays an important role in the prediction of storm surges. The uncertainty of a typhoon’s intensity and track means it is necessary to use an ensemble model to predict typhoon storm surges. A hydrodynamic model, which is operational at the National Marine Environmental Forecasting Center, is applied to conduct surge simulations for South China coastal areas using the best track data with parametric wind and pressure models. The results agree well with tidal gauge observations. To improve the calculation efficiency, the hydrodynamic model is modified using CUDA Fortran. The calculation results are almost the same as those from the original model, but the calculation time is reduced by more than 99%. A total of 150 typhoon cases are generated by combining 50 typhoon tracks from the European Centre for Medium-Range Weather Forecasts with three possible typhoon intensity forecasts. The surge ensembles are computed by the improved hydrodynamic model. Based on the simulated storm surges for the different typhoon cases, ensemble and probability forecast products can be provided. The mean ensemble results and probability forecast products are shown to agree well with the observed storm surge caused by Typhoon Mangkhut. The improved model is highly suitable for ensemble numerical forecasts, providing better forecast products for decision-making, and can be easily implemented to run on regular workstations.

Fast Monte-Carlo Photon Transport Employing GPU-Based Parallel Computation

Monte Carlo (MC) is known to be the most accurate dose calculation method. However, MC suffers from high computational cost as a large number of particles have to be simulated to achieve the desired statistical uncertainty. Enhancing computational power by parallelizing the simulation with multiple GPU threads reduces the time required to reach the desired uncertainty in MC simulation. In this article, we present DOSXYZgpu, a GPU implementation of EGSnrc code which is written in CUDA Fortran as an algorithm. This article relies on a well validated and popular code among medical physicists, EGSnrc/DOSXYZnrc. In order to transport particles between two consecutive interactions, we developed an algorithm to handle several thousands of histories per warp. DOSXYZgpu implementation is evaluated with the original sequential EGSnrc/DOSXYZnrc. Maximum speedup of 205 times is achieved while the statistical uncertainty of the simulation is preserved. The $t$ -test statistical analysis indicates that for more than 95% of the voxels there is no significant difference between the results obtained from the GPU and the CPU.

GPU acceleration of CaNS for massively-parallel direct numerical simulations of canonical fluid flows

This work presents the GPU acceleration of the open-source code CaNS for very fast massively-parallel simulations of canonical fluid flows. The distinct feature of the many-CPU Navier–Stokes solver in CaNS is its fast direct solver for the second-order finite-difference Poisson equation, based on the method of eigenfunction expansions. The solver implements all the boundary conditions valid for this type of problems in a unified framework. Here, we extend the solver for GPU-accelerated clusters using CUDA Fortran. The porting makes extensive use of CUF kernels and has been greatly simplified by the unified memory feature of CUDA Fortran, which handles the data migration between host (CPU) and device (GPU) without defining new arrays in the source code. The overall implementation has been validated against benchmark data for turbulent channel flow and its performance assessed on a NVIDIA DGX-2 system (16 T V100 32Gb, connected with NVLink via NVSwitch). The wall-clock time per time step of the GPU-accelerated implementation is impressively small when compared to its CPU implementation on state-of-the-art many-CPU clusters, as long as the domain partitioning is sufficiently small that the data resides mostly on the GPUs. The implementation has been made freely available and open source under the terms of an MIT license.

A Novel GPU-Based Acceleration Algorithm for a Longwave Radiative Transfer Model

Graphics processing unit (GPU)-based computing for climate system models is a longstanding research area of interest. The rapid radiative transfer model for general circulation models (RRTMG), a popular atmospheric radiative transfer model, can calculate atmospheric radiative fluxes and heating rates. However, the RRTMG has a high calculation time, so it is urgent to study its GPU-based efficient acceleration algorithm to enable large-scale and long-term climatic simulations. To improve the calculative efficiency of radiation transfer, this paper proposes a GPU-based acceleration algorithm for the RRTMG longwave radiation scheme (RRTMG_LW). The algorithm concept is accelerating the RRTMG_LW in the g- p o i n t dimension. After implementing the algorithm in CUDA Fortran, the G-RRTMG_LW was developed. The experimental results indicated that the algorithm was effective. In the case without I/O transfer, the G-RRTMG_LW on one K40 GPU obtained a speedup of 30.98× over the baseline performance on one single Intel Xeon E5-2680 CPU core. When compared to its counterpart running on 10 CPU cores of an Intel Xeon E5-2680 v2, the G-RRTMG_LW on one K20 GPU in the case without I/O transfer achieved a speedup of 2.35×.

Development of GPU-accelerated kinematic wave model using CUDA fortran

Development of GPU-accelerated kinematic wave model using CUDA fortran

High-performance implementation of a Runge–Kutta finite-difference scheme for the Higgs boson equation in the de Sitter spacetime

High performance computations are presented for the Higgs boson equation in the de Sitter Spacetime using explicit fourth order Runge–Kutta scheme on the temporal discretization and fourth order finite difference discretization in space. In addition to the full, (3+1)−dimensional equation we also examine the (1+1)−dimensional radial solutions. The numerical code for the (3+1)−dimensional equation is programmed in CUDA Fortran and is performed on NVIDIA Tesla K40c GPU Accelerators. The radial form of the equation is simulated in MATLAB. The numerical results demonstrate the existing theoretical result that under certain conditions bubbles form in the scalar field. We also demonstrate the known blow-up phenomena for the solutions of the related semilinear Klein–Gordon equation with imaginary mass. Our numerical studies suggest several previously not known properties of the solution for the Higgs boson equation in the de Sitter spacetime for which theoretical proofs do not exist yet: 1. smooth solution exists for all time if the initial conditions are compactly supported and smooth; 2. under some conditions no bubbles form; 3. solutions converge to step functions related to unforced, damped Duffing equations.

AFiD-GPU: A versatile Navier–Stokes solver for wall-bounded turbulent flows on GPU clusters

The AFiD code, an open source solver for the incompressible Navier–Stokes equations (http://www.afid.eu), has been ported to GPU clusters to tackle large-scale wall-bounded turbulent flow simulations. The GPU porting has been carried out in CUDA Fortran with the extensive use of kernel loop directives (CUF kernels) in order to have a source code as close as possible to the original CPU version; just a few routines have been manually rewritten. A new transpose scheme has been devised to improve the scaling of the Poisson solver, which is the main bottleneck of incompressible solvers. For large meshes the GPU version of the code shows good strong scaling characteristics, and the wall-clock time per step for the GPU version is an order of magnitude smaller than for the CPU version of the code. Due to the increased performance and efficient use of memory, the GPU version of AFiD can perform simulations in parameter ranges that are unprecedented in thermally-driven wall-bounded turbulence. To verify the accuracy of the code, turbulent Rayleigh–Bénard convection and plane Couette flow are simulated and the results are in excellent agreement with the experimental and computational data that have been published in literature. Program summaryProgram Title: AFiD-GPUProgram Files doi:http://dx.doi.org/10.17632/rwjdg7ry66.1Licensing provisions: MITProgramming language: Fortran 90, CUDA Fortran, MPIExternal routines: PGI, CUDA Toolkit, FFTW3, HDF5Nature of problem: Solving the three-dimensional Navier–Stokes equations coupled with a scalar field in a cubic box bounded between two walls and with periodic boundary conditions in the horizontal directions.Solution method: Second order finite difference method for spatial discretization, third order Runge–Kutta scheme in combination with Crank–Nicolson for the implicit terms for time advancement, two dimensional pencil distributed MPI parallelization, GPU accelerated routines.Additional comments including restrictions and unusual features: The code is available and supported on https://github.com/PhysicsofFluids/AFiD_GPU_opensource.

A GPU-accelerated implicit meshless method for compressible flows

This paper develops a recently proposed GPU based two-dimensional explicit meshless method (Ma et al., 2014) by devising and implementing an efficient parallel LU-SGS implicit algorithm to further improve the computational efficiency. The capability of the original 2D meshless code is extended to deal with 3D complex compressible flow problems. To resolve the inherent data dependency of the standard LU-SGS method, which causes thread-racing conditions destabilizing numerical computation, a generic rainbow coloring method is presented and applied to organize the computational points into different groups by painting neighboring points with different colors. The original LU-SGS method is modified and parallelized accordingly to perform calculations in a color-by-color manner. The CUDA Fortran programming model is employed to develop the key kernel functions to apply boundary conditions, calculate time steps, evaluate residuals as well as advance and update the solution in the temporal space. A series of two- and three-dimensional test cases including compressible flows over single- and multi-element airfoils and a M6 wing are carried out to verify the developed code. The obtained solutions agree well with experimental data and other computational results reported in the literature. Detailed analysis on the performance of the developed code reveals that the developed CPU based implicit meshless method is at least four to eight times faster than its explicit counterpart. The computational efficiency of the implicit method could be further improved by ten to fifteen times on the GPU.

GPU acceleration of the boundary element method for shear-deformable bending of plates

This paper presents a novel implementation of GPU computing for boundary element method (BEM) formulation of plates. The new GPU code written in CUDA Fortran alters three kernels: the calculation of influence matrices, the solution of equations, and the computation of internal values. Comparisons for computation time and different GPU architecture are presented. The formulation is implemented for both constant and quadratic elements. The efficiency of the parallel quadratic code is demonstrated via analysis of practical building slabs. The benefits of parallel computing of the solution of the system of equations and of points inside the domain are discussed.

Nekbone performance on GPUs with OpenACC and CUDA Fortran implementations

We present a hybrid GPU implementation and performance analysis of Nekbone, which represents one of the core kernels of the incompressible Navier---Stokes solver Nek5000. The implementation is base...

The development of GPU-based parallel PRNG for Monte Carlo applications in CUDA Fortran

The implementation of Monte Carlo simulation on the CUDA Fortran requires a fast random number generation with good statistical properties on GPU. In this study, a GPU-based parallel pseudo random number generator (GPPRNG) have been proposed to use in high performance computing systems. According to the type of GPU memory usage, GPU scheme is divided into two work modes including GLOBAL_MODE and SHARED_MODE. To generate parallel random numbers based on the independent sequence method, the combination of middle-square method and chaotic map along with the Xorshift PRNG have been employed. Implementation of our developed PPRNG on a single GPU showed a speedup of 150x and 470x (with respect to the speed of PRNG on a single CPU core) for GLOBAL_MODE and SHARED_MODE, respectively. To evaluate the accuracy of our developed GPPRNG, its performance was compared to that of some other commercially available PPRNGs such as MATLAB, FORTRAN and Miller-Park algorithm through employing the specific standard tests. The results of this comparison showed that the developed GPPRNG in this study can be used as a fast and accurate tool for computational science applications.

Accelerating the SCE-UA Global Optimization Method Based on Multi-Core CPU and Many-Core GPU

The famous global optimization SCE-UA method, which has been widely used in the field of environmental model parameter calibration, is an effective and robust method. However, the SCE-UA method has a high computational load which prohibits the application of SCE-UA to high dimensional and complex problems. In recent years, the hardware of computer, such as multi-core CPUs and many-core GPUs, improves significantly. These much more powerful new hardware and their software ecosystems provide an opportunity to accelerate the SCE-UA method. In this paper, we proposed two parallel SCE-UA methods and implemented them on Intel multi-core CPU and NVIDIA many-core GPU by OpenMP and CUDA Fortran, respectively. The Griewank benchmark function was adopted in this paper to test and compare the performances of the serial and parallel SCE-UA methods. According to the results of the comparison, some useful advises were given to direct how to properly use the parallel SCE-UA methods.

CUDA Fortranにおける並列リダクションの実装について

CUDA Fortranにおける並列リダクションの実装について

Computational Fluid Dynamics Computations Using a Preconditioned Krylov Solver on Graphical Processing Units

Graphical processing unit (GPU) computation in recent years has seen extensive growth due to advancement in both hardware and software stack. This has led to increase in the use of GPUs as accelerators across a broad spectrum of applications. This work deals with the use of general purpose GPUs for performing computational fluid dynamics (CFD) computations. The paper discusses strategies and findings on porting a large multifunctional CFD code to the GPU architecture. Within this framework, the most compute intensive segment of the software, the BiCGStab linear solver using additive Schwarz block preconditioners with point Jacobi iterative smoothing is optimized for the GPU platform using various techniques in CUDA Fortran. Representative turbulent channel and pipe flow are investigated for validation and benchmarking purposes. Both single and double precision calculations are highlighted. For a modest single block grid of 64 × 64 × 64, the turbulent channel flow computations showed a speedup of about eightfold in double precision and more than 13-fold for single precision on the NVIDIA Tesla GPU over a serial run on an Intel central processing unit (CPU). For the pipe flow consisting of 1.78 × 106 grid cells distributed over 36 mesh blocks, the gains were more modest at 4.5 and 6.5 for double and single precision, respectively.

Rapid calculation of gravity anomalies based on residual node densities and its GPU implementation

Gravity forward modeling is an important part of three dimensional physical property inversion; however, its time-consuming nature restricts its practical application. Several authors have made efforts to develop this procedure and have achieved some success. This paper proposes a new simple and fast approach for computing gravity anomalies at arbitrary points. In our method, the subsurface is divided into a 3D array of rectangular prismatic blocks, each with a constant density. Then the block density model is transformed into a residual node density model. Finally, gravity anomalies at survey points are calculated using the new residual node density-based forward formula. This method is implemented on a CPU–GPU heterogeneous architecture with the CUDA FORTRAN language and achieves more than four times acceleration compared to the classical method in the same computational environment and two orders of magnitude acceleration compared to one CPU processor on synthetic random density model tests.

A case study of CUDA FORTRAN and OpenACC for an atmospheric climate kernel

The porting of a key kernel in the tracer advection routines of the Community Atmosphere Model – Spectral Element (CAM-SE) to use Graphics Processing Units (GPUs) using OpenACC is considered in comparison to an existing CUDA FORTRAN port. The development of the OpenACC kernel for GPUs was substantially simpler than that of the CUDA port. Also, OpenACC performance was about 1.5× slower than the optimized CUDA version. Particular focus is given to compiler maturity regarding OpenACC implementation for modern FORTRAN, and it is found that the Cray implementation is currently more mature than the PGI implementation. Still, for the case that ran successfully on PGI, the PGI OpenACC runtime was slightly faster than Cray. The results show encouraging performance for OpenACC implementation compared to CUDA while also exposing some issues that may be necessary before the implementations are suitable for porting all of CAM-SE. Most notable are that GPU shared memory should be used by future OpenACC implementations and that derived type support should be expanded.

GPU Computing in Bayesian Inference of Realized Stochastic Volatility Model

The realized stochastic volatility (RSV) model that utilizes the realized volatility as additional information has been proposed to infer volatility of financial time series. We consider the Bayesian inference of the RSV model by the Hybrid Monte Carlo (HMC) algorithm. The HMC algorithm can be parallelized and thus performed on the GPU for speedup. The GPU code is developed with CUDA Fortran. We compare the computational time in performing the HMC algorithm on GPU (GTX 760) and CPU (Intel i7-4770 3.4GHz) and find that the GPU can be up to 17 times faster than the CPU. We also code the program with OpenACC and find that appropriate coding can achieve the similar speedup with CUDA Fortran.

도시기상모델 CFD_NIMR의 GP-GPU 실행을 위한 병렬 프로그램의 구현

본 논문은 도시기상모델인 전산유체역학모델(CFD_NIMR)을 GP-GPU에서 실행시키기 위해 CUDA Fortran 병렬프로그램을 구현하였다. GP-GPU는 원래 PCI 카드 형태의 그래픽 처리 장치이지만 저비용, 저전력으로 대량의 계산을 초고속으로 수행할 수 있는 일반 계산 가속기이다. 모델을 단일 Intel XEON 2.0 GHz CPU에서 실행한 결과와 Nvidia Tesla C1060 GPU에서 실행한 성능을 비교하였을 때 GP-GPU에서 15배 정도의 빠른 속도를 보였다. 또한 다중 CPU를 사용한 MPI 병렬프로그램과 비교한 경우에도 GP-GPU에서 보다 더 효율적인 성능을 보였다. 본 논문에서 제시한 프로그램 방식은 유사한 구조를 가진 수치모델을 GP-GPU 병렬 프로그램으로 구현하는데 쉽게 적용할 수 있을 것으로 기대한다. In this paper, we implemented a CUDA Fortran parallel program to run the CFD_NIMR model on GP-GPU's, which simulates air diffusion on urban terrains. A GP-GPU is graphic processing unit in the form of a PCI card, and a general calculation accelerator to perform a large amount of high speed calculations with low cost and electric power. The GP-GPU gives performance enhancement of speed by 15 times to compare the Nvidia Tesla C1060 GPU with Intel XEON 2.0 GHz CPU. In addition, the program on a GP-GPU shows efficient performance compared to an MPI parallel program on multiple CPU's. It is expected that a proposed programming method on the GP-GPU parallel program can be used for numerical models with a similar structure.

Parallelized CCHE2D flow model with CUDA Fortran on Graphics Processing Units

This paper presents the CCHE2D implicit flow model parallelized using CUDA Fortran programming technique on Graphics Processing Units (GPUs). A parallelized implicit Alternating Direction Implicit (ADI) solver using Parallel Cyclic Reduction (PCR) algorithm on GPU is developed and tested. This solver outperforms the Strong Implicit Procedure (SIP) solver and its parallel alternatives. Computing accuracy and efficiency of both CPU and GPU versions of models were compared with one experimental case and one field case. It has been demonstrated that the parallelized CCHE2D flow model with CUDA Fortran is capable of accurately predicting steady flow or unsteady flow with a much higher computing efficiency on the GPU.

WRF 물리 과정의 GP-GPU 계산을 위한 CUDA Fortran 프로그램 구현

We parallelized WRF major physics routines for Nvidia GP-GPUs with CUDA Fortran. GP-GPUs are originally designed for graphic processing, but show high performance with low electricity for calculating numerical models. In the CUDA environment, a data domain is allocated into thread blocks and threads in each thread block are computing in parallel. We parallelized the WRF program to use of thread blocks efficiently. We validated the GP-GPU program with the original CPU program, and the WRF model using GP-GPUs shows efficient speedup.