GPU Memory Access Research Articles

Criticality-aware priority to accelerate GPU memory access

Criticality-aware priority to accelerate GPU memory access

CuWide: Towards Efficient Flow-Based Training for Sparse Wide Models on GPUs

Wide models such as generalized linear models and factorization-based models have been extensively used in various predictive applications, e.g., recommendation, CTR prediction, and image recognition. Due to the memory bounded property of the models, the performance improvement on CPU is reaching the limitation. GPU is known to have many computation units and high memory bandwidth, and becomes a promising platform for training machine learning models. However, the GPU training for the wide models is far from optimal due to the sparsity and irregularity in wide models. The existing GPU-based wide models are even slower than the ones using CPU. The classical training schema of the wide models does not optimized for the GPU architecture, which suffers from large amount of random memory accesses and redundant read/write of intermediate values. In this paper, we propose an efficient GPU-training framework for the large-scale wide models, named cuWide. To fully benefit from the memory hierarchy of GPU, cuWide applies a new flow-based schema for training, which leverages the spatial and temporal locality of wide models to drastically reduce the amount of communication with GPU global memory. To do so, we adopt a bigraph computation model to efficiently realize the flow-based schema and exploit three flexible interfaces for programming. Further, we use the 2D partition of mini-batch (in sample and feature dimensions) with proposed graph abstraction to optimize GPU memory access for sparse data, and apply several spatial-temporal caching mechanisms (importance-based model caching and cross-stage accumulation caching mechanisms) to achieve a high performance kernel. To efficiently implement cuWide, we also propose several GPU-oriented optimizations, including feature-oriented data layout to enhance the data locality, replication mechanism to reduce update conflicts in shared memory, and multi-stream scheduling to overlap data transferring and kernel computing. We show that cuWide can be up to more than 20× faster than the state-of-the-art GPU solutions and multi-core CPU solutions.

A novel GPU-parallelized meshless method for solving compressible turbulent flows

This paper presents a novel GPU-parallelized meshless method for solving Reynolds-averaged Navier–Stokes equations with the Spalart–Allmaras turbulence model. Least-square curve fit is utilized to discretize the spatial derivatives of the equations, and a Roe-type upwind scheme is used for computing the flux terms. The compute unified device architecture (CUDA) Fortran programming model is employed to port the meshless method from CPU to GPU in a way of achieving efficiency. For the extracted GPU parallel tasks, a particular two-dimensional thread hierarchy is designed to construct the corresponding computational kernels. Then, a modified strategy, multi-layered point reordering, and a proposed strategy, shared memory access tuning, are used to manage the GPU memory access. A series of typical two- and three-dimensional test cases, including transonic flows over an aerofoil, a wing or a CRM wing-body combination, were carried out to verify the developed method. The computed results agreed well with experimental data and other numerical solutions reported in literature. Impressive speedups, over 40× and up to 79× with respect to a single threaded CPU implementation, are successfully achieved for the benchmark tests.

Thread Batching for High-performance Energy-efficient GPU Memory Design

Massive multi-threading in GPU imposes tremendous pressure on memory subsystems. Due to rapid growth in thread-level parallelism of GPU and slowly improved peak memory bandwidth, memory becomes a bottleneck of GPU’s performance and energy efficiency. In this article, we propose an integrated architectural scheme to optimize the memory accesses and therefore boost the performance and energy efficiency of GPU. First, we propose a thread batch enabled memory partitioning (TEMP) to improve GPU memory access parallelism. In particular, TEMP groups multiple thread blocks that share the same set of pages into a thread batch and applies a page coloring mechanism to bound each stream multiprocessor (SM) to the dedicated memory banks. After that, TEMP dispatches the thread batch to an SM to ensure high-parallel memory-access streaming from the different thread blocks. Second, a thread batch-aware scheduling (TBAS) scheme is introduced to improve the GPU memory access locality and to reduce the contention on memory controllers and interconnection networks. Experimental results show that the integration of TEMP and TBAS can achieve up to 10.3% performance improvement and 11.3% DRAM energy reduction across diverse GPU applications. We also evaluate the performance interference of the mixed CPU+GPU workloads when they are run on a heterogeneous system that employs our proposed schemes. Our results show that a simple solution can effectively ensure the efficient execution of both GPU and CPU applications.

GPU accelerated interferometric SAR processing for Sentinel-1 TOPS data

Sentinel-1 (S-1) TOPS data are widely applied in InSAR applications to monitor earthquakes and landslides. The large S-1 coverage however, leads to a high computational cost when executing InSAR techniques. Thus, we develop a GPU accelerated S-1 InSAR processing method implemented on a personal desktop. In the proposed method, computationally expensive modules including geometric coregistration, resampling, Enhanced Spectral Diversity, and coherence estimation, are implemented using CUDA. In addition, several optimizations are employed to enhance the efficiency of these modules. We select an efficient approximation method in the geometric coregistration module, and improve the GPU memory access efficiency in the resampling module through the GPU texture memory, temporary register array, and a configuration with more L1 cache. We develop a novel GPU-based parallel coherence estimation algorithm in ESD and coherence estimation modules, and use the asynchronous data transfer technology to hide the costs of CPU-GPU data transfer for resampling, ESD, and coherence estimation modules. After several optimizations, our GPU-accelerated modules (considering CPU-GPU transmission costs) achieves speedup ratios up to 157x, 166x, 145x, and 168x with respect to their single-threaded CPU counterparts. For a full frame S-1 image, our method reduces the computation time from 1415.32s to 8.59s. Moreover, our method is also validated in two case studies of the 2016 Mw6.2 Central Italy earthquake and 2018 Mw6.9 Leilani Estates earthquake caused by the Kilauea eruption in Hawaii.

A multi-layered point reordering study of GPU-based meshless method for compressible flow simulations

In meshless methods, clouds of points irregularly distributed are widely used in discretizing computational domains and are usually unavoidable to accommodate complex geometries. However, the irregularity of points has been reported to be negative effect on the GPU memory access pattern, which results in low performance in GPU computations. In order to remedy this negative effect, a multi-layered point reordering (MLPRO) approach is proposed in this paper for GPU-based meshless implementations. Layer structures based on the virtual connections between central and satellite points in meshless clouds are constructed and used to reorder the points in a layer-by-layer manner. Besides, point reordering inside each thread warp, which is rarely concerned in GPU implementations, is further considered by proposing a supplemental group satellite reordering to form a modified MLPRO approach. Furthermore, by defining virtual connectivity matrixes of meshless clouds of points in the whole computational domain, the effect of reordering mentioned to the data localities can be visibly observed to have a comprehensive view of the improvement of point locality. Supersonic flows in a rectangular channel are firstly selected to test the effect extent of irregularity of meshless points to the GPU performance by increasing the percentage of irregular points occupied in the computational domain. Then flows over two- and three-dimensional aerodynamic configurations are simulated to show the performances of the reordering approaches presented. Numerical results show that significant enhancements of GPU speedups can be achieved for all test cases, particularly for the three-dimensional M6 wing and RAE wing-body combination cases with up to 2.5× further speedups, which is meaningful for simulations with large-scale irregular meshless clouds of points.

A GPU-Aware Parallel Index for Processing High-Dimensional Big Data

The problem of the curse of dimensionality for processing large high-dimensional datasets has been an open challenge. Numerous research efforts have been proposed for improving query performance in high-dimensional space through hierarchical indexing using the R-tree or its variants and exploring parallel processing of the R-tree on GPUs. Despite these existing efforts, the curse of dimensionality remains to be a grand challenge since the existing methods deteriorate drastically as the dimensionality of datasets increases. To cope with this problem, we present a novel GPU-aware parallel indexing method called G-tree, which offers consistent and stable performance in high-dimensional space. The rationale of the G-tree is to combine the efficiency of the R-tree in low-dimensional space with the massive parallel processing potential of GPUs by introducing a new data structure and three new optimization techniques to better utilize the GPU memory structure for accelerating both index search and index node access on GPUs. The first two optimizations promote effective parallelism utilization in GPU memory access. We dedicate the third optimization to further speed up the G-tree index by conducting progressive filtering using our dimension filters. We evaluate the validity of the G-tree approach by extensive experiments on high-dimensional datasets, showing that the G-tree outperforms the existing state-of-the-art techniques.

A GPU-Accelerated Multivoxel Update Scheme for Iterative Coordinate Descent (ICD) Optimization in Statistical Iterative CT Reconstruction (SIR)

Statistical iterative reconstruction (SIR) algorithms have shown great potential for improving image quality in reduced and low dose X-ray computed tomography (CT). However, high computational cost and long reconstruction times have so far prevented the use of SIR in practical applications. Various optimization algorithms have been proposed to make SIR parallelizable for execution on multicore computational platforms, whereas others have sought to improve its convergence rate. Parallelizing on a set of decoupled voxels within an iterative coordinate descent (ICD) optimization framework has shown good promise to achieve both of these premises. However, so far these types of frameworks come at the price of additional complexities or are limited to parallel beam geometry only. We improve on this prior research and present a framework, which also achieves parallelism by processing sets of independent voxels, but does not introduce additional complexities and has no restrictions on beam geometry. Our method uses a novel multivoxel update (MVU) scheme within a general ICD framework fully optimized for acceleration on commodity GPUs. We also investigate different GPU memory access patterns to increase cache hit-rates that result in improved time performance in the ICD framework. Experiments demonstrate speedups of two orders of magnitude for clinical datasets in cone-beam CT geometry, compared to the single-voxel update scheme native to conventional ICD-based SIR. Finally, since our MVU scheme operates on fully independent voxels, it maintains the fast convergence properties of ICD-based SIR. Consequently, the speedups achieved by parallel computing are not diminished by slower convergence of the iterative updates or by any additional overhead to decouple conflicting voxels.

SU‐C‐207B‐01: A Novel Graphics Processing Units (GPU) Implementation of Discrete Wavelet Transformation

Purpose:To design and implement a GPU‐based discrete wavelet transformation (DWT) to be used in medical image reconstruction, image processing, and data compression. DWTs are widely used in medical physics, but the computation of DWTs is time consuming for large volumetric data. An efficient parallel implementation of DWTs is essential for many time sensitive applications, such as 4DCT.Methods:We choose Daubechies wavelet transformations as a benchmark, implementing both DWT and inverse DWT (IDWT). The reference CPU code is from “Numerical Recipes in C”. We implemented GPU‐based codes using C++ with CUDA 7.5. A GPU is specialized processors with a highly parallel structure originally designed for manipulating computer graphics. GPU computation is highly memory‐bounded. To optimize GPU memory access pattern and achieve best performance for transformation along Y‐direction, the data are transposed before and after the DWT (IDWT), which can sharply reduce computing efforts. Both CPU and GPU codes are unit‐tested and the result differences between two implementations are verified to be within rounding error. The hardware platform was a desktop with Intel Xeon E5‐1607 3.00 GHz CPU and NVIDIA Quadro K2000 GPU.Results:With the GPU implemented code, we process a medical CT image size of 512×512×256 in 0.5 seconds, which is 60 times faster than the CPU implementation. A naive non‐optimized GPU implementation by direct parallelization approaches only 10 times speedup for 2‐D and 3‐D DWT and IDWT. In comparison, the optimization of GPU memory access pattern can obtain an extra 6x speedup.Conclusion:We developed an efficient implementation of GPU‐based DWT/IDWT for medical image processing. To achieve the best performance of any medical imaging processing algorithm, developers should pay attention to memory access pattern optimization when implementing on GPU architecture.This project is supported by CPRIT grant under RP150485.

GPU-Accelerated Forward and Back-Projections With Spatially Varying Kernels for 3D DIRECT TOF PET Reconstruction

We describe a GPU-accelerated framework that efficiently models spatially (shift) variant system response kernels and performs forward- and back-projection operations with these kernels for the DIRECT (Direct Image Reconstruction for TOF) iterative reconstruction approach. Inherent challenges arise from the poor memory cache performance at non-axis aligned TOF directions. Focusing on the GPU memory access patterns, we utilize different kinds of GPU memory according to these patterns in order to maximize the memory cache performance. We also exploit the GPU instruction-level parallelism to efficiently hide long latencies from the memory operations. Our experiments indicate that our GPU implementation of the projection operators has slightly faster or approximately comparable time performance than FFT-based approaches using state-of-the-art FFTW routines. However, most importantly, our GPU framework can also efficiently handle any generic system response kernels, such as spatially symmetric and shift-variant as well as spatially asymmetric and shift-variant, both of which an FFT-based approach cannot cope with.

A Generic and Scalable Pipeline for GPU Tetrahedral Grid Rendering

Recent advances in algorithms and graphics hardware have opened the possibility to render tetrahedral grids at interactive rates on commodity PCs. This paper extends on this work in that it presents a direct volume rendering method for such grids which supports both current and upcoming graphics hardware architectures, large and deformable grids, as well as different rendering options. At the core of our method is the idea to perform the sampling of tetrahedral elements along the view rays entirely in local barycentric coordinates. Then, sampling requires minimum GPU memory and texture access operations, and it maps efficiently onto a feed-forward pipeline of multiple stages performing computation and geometry construction. We propose to spawn rendered elements from one single vertex. This makes the method amenable to upcoming Direct3D 10 graphics hardware which allows to create geometry on the GPU. By only modifying the algorithm slightly it can be used to render per-pixel iso-surfaces and to perform tetrahedral cell projection. As our method neither requires any pre-processing nor an intermediate grid representation it can efficiently deal with dynamic and large 3D meshes.