Multi-core Graphics Processing Units Research Articles

Multi-Node Multi-GPU Diffeomorphic Image Registration for Large-Scale Imaging Problems.

We present a Gauss-Newton-Krylov solver for large deformation diffeomorphic image registration. We extend the publicly available CLAIRE library to multi-node multi-graphics processing unit (GPUs) systems and introduce novel algorithmic modifications that significantly improve performance. Our contributions comprise (i) a new preconditioner for the reduced-space Gauss-Newton Hessian system, (ii) a highly-optimized multi-node multi-GPU implementation exploiting device direct communication for the main computational kernels (interpolation, high-order finite difference operators and Fast-Fourier-Transform), and (iii) a comparison with state-of-the-art CPU and GPU implementations. We solve a 2563-resolution image registration problem in five seconds on a single NVIDIA Tesla V100, with a performance speedup of 70% compared to the state-of-the-art. In our largest run, we register 20483 resolution images (25 B unknowns; approximately 152× larger than the largest problem solved in state-of-the-art GPU implementations) on 64 nodes with 256 GPUs on TACC's Longhorn system.

3D PHOVIS: 3D photoacoustic visualization studio

Photoacoustic (PA) imaging (or optoacoustic imaging) is a novel biomedical imaging method in biological and medical research. This modality performs morphological, functional, and molecular imaging with and without labels in both microscopic and deep tissue imaging domains. A variety of innovations have enhanced 3D PA imaging performance and thus has opened new opportunities in preclinical and clinical imaging. However, the 3D visualization tools for PA images remains a challenge. There are several commercially available software packages to visualize the generated 3D PA images. They are generally expensive, and their features are not optimized for 3D visualization of PA images. Here, we demonstrate a specialized 3D visualization software package, namely 3D Photoacoustic Visualization Studio (3D PHOVIS), specifically targeting photoacoustic data, image, and visualization processes. To support the research environment for visualization and fast processing, we incorporated 3D PHOVIS onto the MATLAB with graphical user interface and developed multi-core graphics processing unit modules for fast processing. The 3D PHOVIS includes following modules: (1) a mosaic volume generator, (2) a scan converter for optical scanning photoacoustic microscopy, (3) a skin profile estimator and depth encoder, (4) a multiplanar viewer with a navigation map, and (5) a volume renderer with a movie maker. This paper discusses the algorithms present in the software package and demonstrates their functions. In addition, the applicability of this software to ultrasound imaging and optical coherence tomography is also investigated. User manuals and application files for 3D PHOVIS are available for free on the website (www.boa-lab.com). Core functions of 3D PHOVIS are developed as a result of a summer class at POSTECH, “High-Performance Algorithm in CPU/GPU/DSP, and Computer Architecture.” We believe our 3D PHOVIS provides a unique tool to PA imaging researchers, expedites its growth, and attracts broad interests in a wide range of studies.

Hybrid multi-projection method using sparse approximate inverses on GPU clusters

The state-of-the-art supercomputing infrastructures are equipped with accelerators, such as graphics processing units (GPUs), that operate as coprocessors for each workstation of the distributed memory system. The multi-projection type methods are a class of algebraic domain decomposition methods based on semi-aggregation techniques. The multi-projection type methods have improved convergence behavior, as the number of subdomains increases, due to the corresponding augmentation of the semi-aggregated local linear systems with more coarse components, while the number of fine components is reduced. Moreover, limited amount of communications among the workstations is required by the proposed method. The utilization of the available GPUs allows an increase in the number of subdomains along with finer-grained parallelism, leading to improved performance. A load-balancing algorithm that ensures the concurrency of the computations on multicore processors and GPUs is proposed. Flexible parallel preconditioned Krylov subspace iterative methods enhanced with multi-projection type methods have been designed appropriately in order to have improved performance, compared to CPU-only or GPU-only executions, by exploiting the available CPUs and GPUs of the distributed memory system concurrently. The unsymmetric local linear systems are solved by the preconditioned Bi-Conjugate Gradient STABilized (BiCGSTAB) method enhanced with the modified generic factored approximate sparse inverse preconditioner, whereas the preconditioned conjugate gradient (CG) method along with the symmetric factored approximate sparse inverse preconditioner is used for the symmetric positive definite local coefficient matrices. Numerical results regarding the convergence behavior, the performance, and the scalability of the proposed method for several problems are given.

Strength Check of Aircraft Parts Based on Multi-GPU Clusters for Fast Calculation of Sparse Linear Equations

In order to improve the cost-effectiveness ratio, the next-generation vehicle needs to meet the requirements of reuse, while adopting a lighter structural weight, so it is necessary to realize the strength calculation and condition monitoring of key components in the digital twin. Most of the current monitoring methods are based on the characteristics of various data acquisition systems, but they need the support of a large number of flight data. The disadvantages of the above strategy can be avoided by reducing the structure of aircraft components to a finite element model and quickly checking the key components in the health management system. In order to solve the problem of fast calculation of the finite element model of the key components of the aircraft, a parallel algorithm and framework of large-scale sparse matrix preprocessing conjugate gradient method based on CUDA(Compute Unified Device Architecture) technology is proposed in the multi GPU(Graphics Processing Unit) workstation cluster environment. Once the sparse matrix is too large to be processed in a single workstation, this paper discusses how to realize the optimized data segmentation in the distributed multi-GPU computing environment. For the problem of iterative solution of matrix preprocessing, two preprocessing strategies of matrix bandwidth reduction parallelization and incomplete Cholesky decomposition are proposed, and asynchronous task concurrency and load balancing strategies are designed on the architecture. The calculation of some examples in the standard sparse matrix database shows that the algorithm and architecture proposed in this paper have the ability to solve large-scale sparse matrix quickly and efficiently, and can complete the fast strength verification of vehicle components.

Large–scale phase–field lattice Boltzmann study on the effects of natural convection on dendrite morphology formed during directional solidification of a binary alloy

The effects of natural convection on dendrite morphologies formed during solidification of a three–dimensional (3D) bulk crystal have not been clarified yet. Although numerical simulation is key to solve the problem, it is not straightforward from a computational cost point of view to simulate the growth of multiple dendrites and liquid flow simultaneously. In this study, we numerically investigate the effects of natural convection on the dendrite morphologies formed during 3D upward directional solidification of an Al–3 wt%Cu alloy, providing a comparison with two–dimensional (2D) simulations. Large–scale simulations of a model coupling phase–field and lattice Boltzmann methods are performed by parallel computation using multi graphics processing units on a supercomputer. As a result of a series of simulations where gravity is changed, similar results are obtained in the 3D and 2D simulations: the average primary arm spacing increases as the gravity decreases in the negative region, the large upward flow causes unstable dendrite growth, and the downward flow enhances the growth of secondary arms. On the other hand, because the natural convection caused under gravity is less in the 3D than the 2D case, the effect of natural convection in the 3D bulk is smaller than in the 2D. In addition, due to differences in flow patterns in front of dendrite tips, there are some distinctions in dendrite growth between 2D and 3D cases.

Parallelizing Multiple Flow Accumulation Algorithm using CUDA and OpenACC

Watershed analysis, as a fundamental component of digital terrain analysis, is based on the Digital Elevation Model (DEM), which is a grid (raster) model of the Earth surface and topography. Watershed analysis consists of computationally and data intensive computing algorithms that need to be implemented by leveraging parallel and high-performance computing methods and techniques. In this paper, the Multiple Flow Direction (MFD) algorithm for watershed analysis is implemented and evaluated on multi-core Central Processing Units (CPU) and many-core Graphics Processing Units (GPU), which provides significant improvements in performance and energy usage. The implementation is based on NVIDIA CUDA (Compute Unified Device Architecture) implementation for GPU, as well as on OpenACC (Open ACCelerators), a parallel programming model, and a standard for parallel computing. Both phases of the MFD algorithm (i) iterative DEM preprocessing and (ii) iterative MFD algorithm, are parallelized and run over multi-core CPU and GPU. The evaluation of the proposed solutions is performed with respect to the execution time, energy consumption, and programming effort for algorithm parallelization for different sizes of input data. An experimental evaluation has shown not only the advantage of using OpenACC programming over CUDA programming in implementing the watershed analysis on a GPU in terms of performance, energy consumption, and programming effort, but also significant benefits in implementing it on the multi-core CPU.

Implementation of the CPU/GPU hybrid parallel method of characteristics neutron transport calculation using the heterogeneous cluster with dynamic workload assignment

In recent years, graphics processing units (GPUs) have been adopted in many High-Performance Computing (HPC) systems due to their massive computational power and superior energy efficiency. And accelerating CPU-version computational code on heterogeneous clusters with multi-core CPUs and GPUs has attracted a lot of attention. One of the focus on heterogeneous computing is to efficiently take advantage of all computational resources, including both CPU and GPU available on a cluster. In this paper, a heterogeneous MPI + OpenMP/CUDA parallel algorithm for solving the 2D neutron transport equation with the method of characteristic (MOC) is implemented. In this algorithm, the spatial domain decomposition technique provides the coarse-grained parallelism with the MPI protocol while the fine-grained parallelism is exploited through OpenMP (in CPU calculated domain) and CUDA (in GPU calculated domain) based on the ray parallelization. In order to efficiently leverage the computing power of heterogeneous clusters, a dynamic workload assignment scheme is proposed, which is to distribute the workload based on the runtime performance of CPUs and GPUs in the cluster. Moreover, the strong scaling performance of the MPI + CUDA parallelization is studied through a performance analysis model which provides the detailed impact of the degradation in iteration scheme, the load imbalance issue, the data copy between CPUs and GPUs, and the MPI communication in the MPI + CUDA parallel algorithm. And the corresponding conclusion is still tenable for the MPI + OpenMP/CUDA parallelization. The C5G7 2D benchmark and an extended 2D whole-core problem are calculated with MPI + CUDA parallelization, MPI + OpenMP/CUDA parallelization, and the MPI parallelization for comparison. Numerical results demonstrate that the heterogeneous parallel algorithm maintains the desired accuracy. And the dynamic workload assignment scheme can provide the optimal workload assignment which ideally matches the experimental results. In addition, over 11% improvement is observed in MPI + OpenMP/CUDA parallelization compared against the MPI + CUDA parallelization. Moreover, the CPUs/GPUs heterogeneous clusters significantly outperform the CPUs clusters and one heterogeneous node shows basically five times faster than a CPUs node.

GPU-Based Lossless Compression of Aurora Spectral Data using Online DPCM

It is well known that aurorae have very high research value, but the data volume of aurora spectral data is very large, which brings great challenges to storage and transmission. To alleviate this problem, compression of aurora spectral data is indispensable. This paper presents a parallel Compute Unified Device Architecture (CUDA) implementation of the prediction-based online Differential Pulse Code Modulation (DPCM) method for the lossless compression of the aurora spectral data. Two improvements are proposed to improve the compression performance of the online DPCM method. One is on the computing of the prediction coefficients, and the other is on the encoding of the residual. In the CUDA implementation, we proposed a decomposition method for the matrix multiplication to avoid redundant data accesses and calculations. In addition, the CUDA implementation is optimized with a multi-stream technique and multi-graphics processing unit (GPU) technique, respectively. Finally, the average compression time of an aurora spectral image reaches about 0.06 s, which is much less than the 15 s aurora spectral data acquisition time interval and can save a lot of time for transmission and other subsequent tasks.

Decentralized Equalization With Feedforward Architectures for Massive MU-MIMO

Linear data-detection algorithms that build on zero forcing (ZF) or linear minimum mean-square error (L-MMSE) equalization achieve near-optimal spectral efficiency in massive multi-user multiple-input multiple-output (MU-MIMO) systems. Such algorithms, however, typically rely on centralized processing at the base-station (BS) which results in (i) excessive interconnect and chip input/output (I/O) data rates and (ii) high computational complexity. Decentralized baseband processing (DBP) partitions the BS antenna array into independent clusters that are associated with separate radio-frequency circuitry and computing fabrics in order to overcome the limitations of centralized processing. In this paper, we investigate decentralized equalization with feedforward architectures that minimize the latency bottlenecks of existing DBP solutions. We propose two distinct architectures with different interconnect and I/O bandwidth requirements that fuse the local equalization results of each cluster in a feedforward network. For both architectures, we consider maximum ratio combining, ZF, L-MMSE, and a nonlinear equalization algorithm that relies on approximate message passing, and we analyze the associated post-equalization signal-to-noise-and-interference-ratio (SINR). We provide reference implementation results on a multi graphics processing unit (GPU) system which demonstrate that decentralized equalization with feedforward architectures enables throughputs in the Gb/s regime and incurs no or only a small performance loss compared to centralized solutions.

基于多图像处理单元的Matlab计算全息图快速算法

基于多图像处理单元的Matlab计算全息图快速算法

Fast parallel beam propagation method based on multi-core and many-core architectures

In this paper, a fast technique is suggested to accelerate the computation of the fast Fourier transform beam propagation method (FFT-BPM). The FFT-BPM is executed on a graphical processing unit (GPU) and multi-core processor GPUs to speed up the computation of huge number of propagation steps with a higher speed than the traditional CPU. Further, the suggested technique is implemented in parallel approach which is faster than serial implementation. The achieved speedup factor is 150× and 5× using GPU and eight cores multiprocessor, respectively with respect to a single core processing time of 215 steps input Gaussian beam. In order to verify the speed of the proposed technique, the possibility of using the BPM to compute the time-consuming Goos–Hänchen shift calculation is proposed. Further, the propagation of a single mode light beam in fiber optic for 5 × 106 steps is executed using GPU. It is found that the speed up of the studied mode is equal to 168x over a single core calculation.

Multi-architecture Monte-Carlo (MC) simulation of soft coarse-grained polymeric materials: SOft coarse grained Monte-Carlo Acceleration (SOMA)

Multi-component polymer systems are important for the development of new materials because of their ability to phase-separate or self-assemble into nano-structures. The Single-Chain-in-Mean-Field (SCMF) algorithm in conjunction with a soft, coarse-grained polymer model is an established technique to investigate these soft-matter systems. Here we present an implementation of this method: SOft coarse grained Monte-Carlo Acceleration (SOMA). It is suitable to simulate large system sizes with up to billions of particles, yet versatile enough to study properties of different kinds of molecular architectures and interactions.We achieve efficiency of the simulations commissioning accelerators like GPUs on both workstations as well as supercomputers. The implementation remains flexible and maintainable because of the implementation in the scientific programming language C enhanced by OpenACC pragmas for the accelerators.We present implementation details and features of the program package, investigate the scalability of our implementation SOMA, and discuss two applications, which cover system sizes that are difficult to reach with other, common particle-based simulation methods. Program summaryProgram Title: SOMAProgram Files doi:http://dx.doi.org/10.17632/j3thz43k93.1Licensing provisions: GNU Lesser General Public License version 3Programming language: C99, OpenACC, OpenMP, MPI, pythonNature of problem: Efficient simulation of polymer materials, their phase-separation or self-assembly using a highly coarse-grained, soft, particle-based model [1]. The simulations help predicting self-assembled structures that, for example, find application in the fabrication of large-scale, dense arrays of nano-structures by Directed Self-Assembly (DSA).Solution method: Representation of soft, non-bonded interactions by quasi-instantaneous fields on a collocation grid using the Single-Chain-in-Mean-Field (SCMF) [2] algorithm, and sampling of configuration space using local random Monte-Carlo (MC) displacements and Smart Monte-Carlo (SMC). Parallelization using MPI and accelerators such as Graphics Processing Units (GPUs).Restrictions: The program has not been tested for more than 10 billion particles.Unusual Features: Efficient simulation on different hardware architectures and accelerators, including multi-core Central Processing Units (CPUs) and GPUs. Furthermore, it is possible to combine different architectures within a single simulation.[1] M. Müller, J. Stat. Phys. 145 (2011) 967[2] K. C. Daoulas, M. Müller, J. Chem. Phys. 125 (2006) 184904

FaST-LMM for Two-Way Epistasis Tests on High-Performance Clusters.

We introduce a version of the epistasis test in FaST-LMM for clusters of multithreaded processors. This new software maintains the sensitivity of the original FaST-LMM while delivering acceleration that is close to linear on 12-16 nodes of two recent platforms, with respect to improved implementation of FaST-LMM presented in an earlier work. This efficiency is attained through several enhancements on the original single-node version of FaST-LMM, together with the development of a message passing interface (MPI)-based version that ensures a balanced distribution of the workload as well as a multigraphics processing unit (GPU) module that can exploit the presence of multiple GPUs per node.

Grain growth kinetics in submicrometer-scale molecular dynamics simulation

Grain growth kinetics under the anisotropic grain boundary properties is investigated by large-scale and long-time molecular dynamics (MD) simulations of contentious processes of nucleation, solidification and grain growth in a submicrometer-scale system. Microstructures obtained via homogeneous nucleation from undercooled melt iron consists of approximately 1500 grains and the number of grains decreases to one tenth of the number via the grain growth process. The grain growth exponent obtained from the MD simulation deviates from the ideal value since anisotropic effects in the grain boundary properties are inherently included in MD simulations. It is confirmed that the decrease of the reduced mobility (i.e., the product of the intrinsic grain boundary mobility and the grain boundary energy) is a dominant factor for the deviation from the ideal grain growth. The deviation from the Mackenzie function for the distribution of the disorientation angle between neighboring grains implies that the preferential selection of grain boundaries with small grain boundary energies occurs during the grain growth. This enhances the anisotropy in grain boundary properties and therefore decreases the reduced mobility of the grain boundary. Moreover, a multi-phase-field simulation starting from a MD configuration results in an ideal grain growth when a constant value of the reduced mobility is employed, which validates the discussion on the reduced mobility. The new insight in this study is achieved for the first time owing to a multi-graphics processing unit (GPU) parallel computation over 50 days for one case using 128 GPUs on the GPU-rich supercomputer.

Parallelizing maximum likelihood classification (MLC) for supervised image classification by pipelined thread approach through high-level synthesis (HLS) on FPGA cluster

ABSTRACTHigh spectral, spatial, vertical and temporal resolution data are increasingly available and result in the serious challenge to process big remote-sensing images effectively and efficiently. This article introduced how to conduct supervised image classification by implementing maximum likelihood classification (MLC) over big image data on a field programmable gate array (FPGA) cloud. By comparing our prior work of implementing MLC on conventional cluster of multicore computers and graphics processing unit, it can be concluded that FPGAs can achieve the best performance in comparison to conventional CPU cluster and K40 GPU, and are more energy efficient. The proposed pipelined thread approach can be extended to other image-processing solutions to handle big data in the future.

Study of infrared reflection characteristics of aerial target using MODIS data on GPU

To study of the infrared signature of an aerial target, it is required to precisely model the background radiation. Simple empirical models or standard atmospheric models in LOWTRAN/MODTRAN were used in earlier studies. To further precisely model the thermal radiation of earth’s surface and atmospheric radiance/transmittance, the atmospheric profile, land surface temperature, and emissivity, the sea surface temperature retrieved from moderate-resolution imaging spectroradiometer, and the sea surface emissivity model developed by Wu and Smith are utilized in this study. Meanwhile, considering that the reflection of background radiation incident from different directions in each spectral wavelength can be calculated in parallel, implementations using open multi-processing and compute unified device architecture on a multi-core CPU and many-core graphics processing unit (GPU) are presented and speedups of 9 $$\times$$ and 258 $$\times$$ are obtained on a platform with dual Xeon E5-2652 CPU and an NVIDIA Tesla K80 GPU card, respectively.

A (multi) GPU iterative reconstruction algorithm based on Hessian penalty term for sparse MRI

A recent trend in the Magnetic Resonance Imaging (MRI) research field is to design and adopt machines that are able to acquire undersampled clinical data, reducing the time for which the patient is lying in the body scanner. Unfortunately, the missing information in these undersampled acquired datasets leads to artefacts in the reconstructed image; therefore, computationally expensive image reconstruction techniques are required. In this paper, we present an iterative regularisation strategy with a second-order derivative penalty term for the reconstruction of undersampled image datasets. Moreover, we compare this approach with other constrained minimisation methods, resulting in improved accuracy. Finally, an implementation on a massively parallel architecture environment, a multi Graphics Processing Unit (GPU) system, of the proposed iterative algorithm is presented. The resulting performance gives clinically-feasible reconstruction run times, speed-up and improvements in terms of reconstruction accuracy of the undersampled MRI images.

A (multi) GPU iterative reconstruction algorithm based on Hessian penalty term for sparse MRI

A recent trend in the Magnetic Resonance Imaging (MRI) research field is to design and adopt machines that are able to acquire undersampled clinical data, reducing the time for which the patient is lying in the body scanner. Unfortunately, the missing information in these undersampled acquired datasets leads to artefacts in the reconstructed image; therefore, computationally expensive image reconstruction techniques are required. In this paper, we present an iterative regularisation strategy with a second-order derivative penalty term for the reconstruction of undersampled image datasets. Moreover, we compare this approach with other constrained minimisation methods, resulting in improved accuracy. Finally, an implementation on a massively parallel architecture environment, a multi Graphics Processing Unit (GPU) system, of the proposed iterative algorithm is presented. The resulting performance gives clinically-feasible reconstruction run times, speed-up and improvements in terms of reconstruction accuracy of the undersampled MRI images.

Multi-GPU configuration of 4D intensity modulated radiation therapy inverse planning using global optimization

We report on the design, implementation and characterization of a multi-graphic processing unit (GPU) computational platform for higher-order optimization in radiotherapy treatment planning. In collaboration with a commercial vendor (Varian Medical Systems, Palo Alto, CA), a research prototype GPU-enabled Eclipse (V13.6) workstation was configured. The hardware consisted of dual 8-core Xeon processors, 256 GB RAM and four NVIDIA Tesla K80 general purpose GPUs. We demonstrate the utility of this platform for large radiotherapy optimization problems through the development and characterization of a parallelized particle swarm optimization (PSO) four dimensional (4D) intensity modulated radiation therapy (IMRT) technique. The PSO engine was coupled to the Eclipse treatment planning system via a vendor-provided scripting interface. Specific challenges addressed in this implementation were (i) data management and (ii) non-uniform memory access (NUMA). For the former, we alternated between parameters over which the computation process was parallelized. For the latter, we reduced the amount of data required to be transferred over the NUMA bridge. The datasets examined in this study were approximately 300 GB in size, including 4D computed tomography images, anatomical structure contours and dose deposition matrices. For evaluation, we created a 4D-IMRT treatment plan for one lung cancer patient and analyzed computation speed while varying several parameters (number of respiratory phases, GPUs, PSO particles, and data matrix sizes). The optimized 4D-IMRT plan enhanced sparing of organs at risk by an average reduction of in maximum dose, compared to the clinical optimized IMRT plan, where the internal target volume was used. We validated our computation time analyses in two additional cases. The computation speed in our implementation did not monotonically increase with the number of GPUs. The optimal number of GPUs (five, in our study) is directly related to the hardware specifications. The optimization process took 35 min using 50 PSO particles, 25 iterations and 5 GPUs.

Adaptive Particle Swarm Optimization with Heterogeneous Multicore Parallelism and GPU Acceleration

Much progress has recently been made in global optimization, with particular attention devoted to robust nature-inspired stochastic methods for difficult, high-dimensional problems. This paper presents a computational study of an adaptation of one such method, particle swarm optimization (PSO), which is analyzed for parallelization on readily-available heterogeneous parallel computational hardware: specifically, multicore technologies accelerated by graphics processing units (GPUs), as well as Intel Xeon Phi co-processors accelerated with vectorization. In this heterogeneous approach, computationally-intensive, task-parallel components are performed with multicore parallelism and data-parallel elements are executed via co-processing (GPUs or vectorization). A computationally intensive adaptive PSO technique is parallelized according to this schema. In experiments with two high-dimensional and complex functions, large speedups can be obtained. Thus, a heterogeneous approach mitigates the time complexity of PSO adaptations, suggesting that other time-intensive stochastic methods can also benefit from the techniques proposed here.