Graphics Processing Units Kernels Research Articles

A static analytical performance model for GPU kernel

Graphics processing units (GPUs) have shown increased popularity and play an important role as kind of coprocessor in heterogeneous co-processing environment. Heavily data parallel problems can be solved efficiently due to tens of thousands threads collaborative work in parallel GPU architecture. The achieved performance, therefore, depends on the capability of multiple threads in parallel collaboration. This paper, a static analytical kernel performance model (SAKP) was proposed to estimate the execution time of GPU kernel. Especially, a set of kernel and device features for target GPU is generated in the proposed model. We determine the performance limiting factors and generate an estimation of kernel execution time with this model. Matrix multiplication (MM) and histogram generation (HG) in NVIDIA GTX680 GPU card were performed to verify our proposed model and showed an absolute error in prediction less than 6.8%.

ZeroKernel: Secure Context-isolated Execution on Commodity GPUs

In the last decade, the dedicated graphics processing unit (GPU) has emerged as an architecture for high-performance computing workloads. Recently, researchers have also focused on the isolation property of a dedicated GPU and suggested GPU-based secure computing environments with several promising applications. However, despite the security analysis conducted by the prior studies, it has been unclear whether a dedicated GPU can be leveraged as a secure processor in the presence of a kernel-privileged attacker. In this paper, we first demonstrate the security of dedicated GPUs through comprehensive studies on context information for GPU execution. The paper shows that a kernel-privileged attacker can manipulate the GPU contexts to redirect memory accesses or execute arbitrary GPU codes on the running GPU kernel. Based on the security analysis, this paper proposes a new on-chip execution model for the dedicated GPU and a novel defense mechanism supporting the security of the on-chip execution. With comprehensive evaluation, the paper assures that the proposed solutions effectively isolate sensitive data in on-chip storages and defend against known attack vectors from a privileged attacker, supporting that the commodity GPUs can be leveraged as a secure processor.

Metric Selection for GPU Kernel Classification

Graphics Processing Units (GPUs) are vastly used for running massively parallel programs. GPU kernels exhibit different behavior at runtime and can usually be classified in a simple form as either “compute-bound” or “memory-bound.” Recent GPUs are capable of concurrently running multiple kernels, which raises the question of how to most appropriately schedule kernels to achieve higher performance. In particular, co-scheduling of compute-bound and memory-bound kernels seems promising. However, its benefits as well as drawbacks must be determined along with which kernels should be selected for a concurrent execution. Classifying kernels can be performed online by instrumentation based on performance counters. This work conducts a thorough analysis of the metrics collected from various benchmarks from Rodinia and CUDA SDK. The goal is to find the minimum number of effective metrics that enables online classification of kernels with a low overhead. This study employs a wrapper-based feature selection method based on the Fisher feature selection criterion. The results of experiments show that to classify kernels with a high accuracy, only three and five metrics are sufficient on a Kepler and a Pascal GPU, respectively. The proposed method is then utilized for a runtime scheduler. The results show an average speedup of 1.18× and 1.1× compared with a serial and a random scheduler, respectively.

Efficient Cache Performance Modeling in GPUs Using Reuse Distance Analysis

Reuse distance analysis (RDA) is a popular method for calculating locality profiles and modeling cache performance. The present article proposes a framework to apply the RDA algorithm to obtain reuse distance profiles in graphics processing unit (GPU) kernels. To study the implications of hardware-related parameters in RDA, two RDA algorithms were employed, including a high-level cache-independent RDA algorithm, called HLRDA, and a detailed RDA algorithm, called DRDA. DRDA models the effects of reservation fails in cache blocks and miss status holding registers to provide accurate cache-related performance metrics. In this case, the reuse profiles are cache-specific. In a selection of GPU kernels, DRDA obtained the L1 miss-rate breakdowns with an average error of 3.86% and outperformed the state-of-the-art RDA in terms of accuracy. In terms of performance, DRDA is 246,000× slower than the real GPU executions and 11× faster than GPGPU-Sim. HLRDA ignores the cache-related parameters and its obtained reuse profiles are general, which can be used to calculate miss rates in all cache sizes. Moreover, the average error incurred by HLRDA was 16.9%.

Analysis and Design Techniques towards High-Performance and Energy-Efficient Dense Linear Solvers on GPUs

Graphics Processing Units (GPUs) are widely used in accelerating dense linear solvers. The matrix factorizations, which dominate the runtime for these solvers, are often designed using a hybrid scheme, where GPUs perform trailing matrix updates, while the CPUs perform the panel factorizations. Consequently, hybrid solutions require high-end CPUs and optimized CPU software in order to deliver high performance. Furthermore, they lack the energy efficiency inherent for GPUs due to the use of less energy-efficient CPUs, as well as CPU-GPU communications. This paper presents analysis and design techniques that overcome the shortcomings of the hybrid algorithms, and allow the design of high-performance and energy-efficient dense LU and Cholesky factorizations that use GPUs only. The full GPU solution eliminates the need for a high-end CPU and optimized CPU software, which leads to a better energy efficiency. We discuss different design choices, and introduce optimized GPU kernels for panel factorizations. The developed solutions achieve <inline-formula><tex-math notation="LaTeX">$90+$</tex-math></inline-formula> percent of the performance of optimized hybrid solutions, while improving the energy efficiency by 50 percent. They outperform the vendor library by 30-50 percent in single precision, and 15-50 percent in double precision. We also show that hybrid designs trail the proposed solutions in performance when optimized CPU software is not available.

Tree-based mesh-refinement GPU-accelerated tsunami simulator for real-time operation

Abstract. This paper presents a fast and accurate tsunami real-time operational model to compute across ocean-wide simulations completely on GPU (graphics processing unit). The spherical shallow water equations are solved using the method of characteristics and upwind cubic interpolation to provide high accuracy and stability. A customized, user interactive, tree-based mesh-refinement method is implemented based on distance from the coast and focal areas to generate a memory-efficient domain with resolutions of up to 50 m. Three specialized and optimized GPU kernels (Wet, Wall and Inundation) are developed to compute the domain block mesh. Multi-GPU is used to further speed up the computation, and a weighted Hilbert space-filling curve is used to produce a balanced workload. Hindcasting of the 2004 Indonesian tsunami is presented to validate and compare the agreement of the arrival times and main peaks at several gauges. Inundation maps are also produced for Kamala and Hambantota to validate the accuracy of our model. Test runs on three Tesla P100 cards on Tsubame 3.0 could fully simulate 10 h in just under 10 min wall-clock time.

A Quantitative Evaluation of Contemporary GPU Simulation Methodology

Contemporary Graphics Processing Units (GPUs) are used to accelerate highly parallel compute workloads. For the last decade, researchers in academia and industry have used cycle-level GPU architecture simulators to evaluate future designs. This paper performs an in-depth analysis of commonly accepted GPU simulation methodology, examining the effect both the workload and the choice of instruction set architecture have on the accuracy of a widely-used simulation infrastructure, GPGPU-Sim. We analyze numerous aspects of the architecture, validating the simulation results against real hardware. Based on a characterized set of over 1700 GPU kernels, we demonstrate that while the relative accuracy of compute-intensive workloads is high, inaccuracies in modeling the memory system result in much higher error when memory performance is critical. We then perform a case study using a recently proposed GPU architecture modification, Cache-Conscious Wavefront Scheduling. The case study demonstrates that the cross-product of workload characteristics and instruction set architecture choice can affect the predicted efficacy of the technique.

A Quantitative Evaluation of Contemporary GPU Simulation Methodology

Contemporary Graphics Processing Units (GPUs) are used to accelerate highly parallel compute workloads. For the last decade, researchers in academia and industry have used cycle-level GPU architecture simulators to evaluate future designs. This paper performs an in-depth analysis of commonly accepted GPU simulation methodology, examining the effect both the workload and the choice of instruction set architecture have on the accuracy of a widely-used simulation infrastructure, GPGPU-Sim. We analyze numerous aspects of the architecture, validating the simulation results against real hardware. Based on a characterized set of over 1700 GPU kernels, we demonstrate that while the relative accuracy of compute-intensive workloads is high, inaccuracies in modeling the memory system result in much higher error when memory performance is critical. We then perform a case study using a recently proposed GPU architecture modification, demonstrating that the cross-product of workload characteristics and instruction set architecture choice can have an affect on the predicted efficacy of the technique.

Efficient GPU-Based Electromagnetic Transient Simulation for Power Systems With Thread-Oriented Transformation and Automatic Code Generation

Electromagnetic transients (EMT) simulation is the most accurate and intensive computation for power systems. Past research has shown the potential of accelerating such simulations using graphics processing units (GPUs). In this paper, an efficient GPU-based parallel EMT simulator is designed. Thread-oriented model transformations are first proposed for the electrical and control systems. Following the transformations, the electrical system is represented by connected networks of massive primitive electrical elements, the computations of which can be constructed as massive fused multiply-add operations and solutions to a linear equation. The control systems are represented by a layered directed acyclic graph with primitive control elements that can be dealt with using single-instruction-multiple-threads groups. Finally, code automation tools are designed to form the GPU kernels. Compared with past work, the proposed model transformations improve the degree of parallelism. Most importantly, the code automation tools improve computational efficiency by substantially reducing addressing and memory access, and render the implementation of the algorithm more general and convenient. Test systems of different sizes were created by connecting multiple IEEE 33-bus distribution systems and adding distributed generators. Simulations were performed on NVIDIA’s K20 $\times$ and P100 cards. The results indicate that the proposed method significantly accelerates EMT simulations compared with a CPU-based program. Real-time performance was also achieved under certain conditions.

GPGPU Power Estimation with Core and Memory Frequency Scaling

With the increasing installation of Graphics Processing Units (GPUs) in supercomputers and data centers, their huge electricity cost brings new environmental and economic concerns. Although Dynamic Voltage and Frequency Scaling (DVFS) techniques have been successfully applied on traditional CPUs to reserve energy, the impact of GPU DVFS on application performance and power consumption is not yet fully understood, mainly due to the complicated GPU memory system. This paper proposes a fast prediction model based on Support Vector Regression (SVR), which can estimate the average runtime power of a given GPU kernel using a set of profiling parameters under different GPU core and memory frequencies. Our experimental data set includes 931 samples obtained from 19 GPU kernels running on a real GPU platform with the core and memory frequencies ranging between 400MHz and 1000MHz. We evaluate the accuracy of the SVR-based prediction model by ten-fold cross validation. We achieve greater accuracy than prior models, being Mean Square Error (MSE) of 0.797 Watt and Mean Absolute Percentage Error (MAPE) of 3.08% on average. Combined with an existing performance prediction model, we can find the optimal GPU frequency settings that can save an average of 13.2% energy across those GPU kernels with no more than 10% performance penalty compared to applying the default setting.

Fully GPU‐based electromagnetic transient simulation considering large‐scale control systems for system‐level studies

As more generators and loads are integrated by power electronic converters with complicated controls, electromagnetic transients (EMTs) simulation becomes an important tool for studying dynamic characteristics of large-scale power systems. To accelerate system-level EMT simulations, a fine-grained parallel algorithm on graphics processing units (GPU) is proposed. By decomposing the computational models of the EMT simulation into heterogeneous, homogeneous and network solution computations, the simulations are mapped into three unified GPU kernels. To incorporate control signals and non-linear features of electrical components, heterogeneous computations are formulated as layered direct acyclic graphs (LDAG) of primitive operations. An LDAG kernel is designed to carry out theses primitive operations efficiently by grouped threads. Then, homogeneous computations for state updates of electrical components are modelled as sets of fused multiply-add (FMA) operations, which are concurrently processed by an FMA kernel. Moreover, a hybrid network solution kernel is designed to solve the network equations, which can adaptively select dense or sparse solvers. Large-scale test systems are created and simulated on an NVIDIA K20x GPU. The results show that the proposed GPU-based EMT simulations are accurate and achieve 10x speedups over the CPU-based ones.

Access Pattern-Aware Cache Management for Improving Data Utilization in GPU

Long latency of memory operation is a prominent performance bottleneck in graphics processing units (GPUs). The small data cache that must be shared across dozens of warps (a collection of threads) creates significant cache contention and premature data eviction. Prior works have recognized this problem and proposed warp throttling which reduces the number of active warps contending for cache space. In this paper we discover that individual load instructions in a warp exhibit four different types of data locality behavior: (1) data brought by a warp load instruction is used only once, which is classified as streaming data (2) data brought by a warp load is reused multiple times within the same warp, called intra-warp locality (3) data brought by a warp is reused multiple times but across different warps, called inter-warp locality (4) and some data exhibit both a mix of intra- and inter-warp locality. Furthermore, each load instruction exhibits consistently the same locality type across all warps within a GPU kernel. Based on this discovery we argue that cache management must be done using per-load locality type information, rather than applying warp-wide cache management policies. We propose Access Pattern-aware Cache Management (APCM), which dynamically detects the locality type of each load instruction by monitoring the accesses from one exemplary warp. APCM then uses the detected locality type to selectively apply cache bypassing and cache pinning of data based on load locality characterization. Using an extensive set of simulations we show that APCM improves performance of GPUs by 34% for cache sensitive applications while saving 27% of energy consumption over baseline GPU.

Energy‐efficient fuzzy control model for GPU‐accelerated packet classification

SummaryAs a core component of many network infrastructures, packet classification requires matching packet headers against a series of predefined rules. Its performance determines, to some extent, how fast packets can be processed. There already exists many proposals, which optimize the throughput of packet classification, but few of them take power consumption into account. To meet the requirements of green network computing, this paper focuses on energy‐efficient solutions that provide reasonable throughput as well. Similar to recent advancements, the graphics processing unit (GPU) is adopted to accelerate rule matching. Then, inspired by the frequency‐variable energy‐consuming model for air conditioners, a fuzzy control–based energy efficiency optimizing model is proposed for GPU‐accelerated packet classification. As demonstrated in the evaluation experiments, when the GPU is in the idle status, the proposed model can save 10 W. In running status, the fuzzy control–based energy efficiency optimizing model can avoid GPU shutdown issue caused by GPU self‐protection mechanism when the GPU temperature rises to 95°C. Furthermore, by improving the resource configuration of GPU kernels according to the model, the overall energy efficiency is enhanced by up to 15.5%, while simultaneously keeping throughput at the same level.

A Hoare Logic for GPU Kernels

We study a Hoare Logic to reason about parallel programs executed on graphics processing units (GPUs), called GPU kernels. During the execution of GPU kernels, multiple threads execute in lockstep, that is, execute the same instruction simultaneously. When the control branches, the two branches are executed sequentially, but during the execution of each branch only those threads that take it are enabled; after the control converges, all the threads are enabled and again execute in lockstep. In this article, we first consider a semantics in which all threads execute in lockstep (this semantics simplifies the actual execution model of GPUs) and adapt Hoare Logic to this setting by augmenting the usual Hoare triples with an additional component representing the set of enabled threads. It is determined that the soundness and relative completeness of the logic do not hold for all programs; a difficulty arises from the fact that one thread can invalidate the loop termination condition of another thread through shared memory. We overcome this difficulty by identifying an appropriate class of programs for which the soundness and relative completeness hold. Additionally, we discuss thread interleaving, which is present in the actual execution of GPUs but not in the lockstep semantics mentioned above. We show that if a program is race free, then the lockstep and interleaving semantics produce the same result. This implies that our logic is sound and relatively complete for race-free programs, even if the thread interleaving is taken into account.

A Comparative Study of Programming Environments Exploiting Heterogeneous Systems

This paper compares programming environments that exploit heterogeneous systems to process a large amount of data efficiently. Our motivation is to investigate the feasibility of the adaptive, transparent migration of intensive computation for a large amount of data across heterogeneous programming languages and processors for high performance and programmability. We compare a variety of programming environments composed of programming languages, such as Java and C, memory space models, such as distinct and shared memory, and parallel processors, such as general-purpose CPUs and graphics processing units (GPUs) to examine their performance-programmability tradeoffs. In addition, we introduce a software-based shared virtual memory that creates a view of the host memory inside GPU kernels to enable seamless computation offloading from the host to the device. This paper reveals a programmability-performance hierarchy in which programs increase their performance at the cost of decreasing programmability. The experimental results suggest the desirability of a well-balanced system.

Systematic detection of memory related performance bottlenecks in GPGPU programs

Abstract Graphics processing units (GPUs) pose an attractive choice for designing high-performance and energy-efficient software systems. This is because GPUs are capable of executing massively parallel applications. However, the performance of GPUs is limited by the contention in memory subsystems, often resulting in substantial delays and effectively reducing the parallelism. In this paper, we propose GRAB, an automated debugger to aid the development of efficient GPU kernels. GRAB systematically detects, classifies and discovers the root causes of memory-performance bottlenecks in GPUs. We have implemented GRAB and evaluated it with several open-source GPU kernels, including two real-life case studies. We show the usage of GRAB through improvement of GPU kernels on a real NVIDIA Tegra K1 hardware – a widely used GPU for mobile and handheld devices. The guidance obtained from GRAB leads to an overall improvement of up to 64%.

A high-throughput DPI engine on GPU via algorithm/implementation co-optimization

The Graphics Processing Unit (GPU) is a promising platform to implement Deep Packet Inspection (DPI) due to the GPU’s rich parallelism and programmability for high performance and frequent pattern update requirements. However, it is a great challenge to achieve a high performance implementation due to the GPU’s performance sensitivity to algorithm and implementation issues such as memory overhead, thread divergence, and large lookup table sizes. In this paper, we propose algorithm and implementation co-optimization techniques that achieve high performance by reducing required memory, removing thread divergence, optimizing memory access patterns, and optimizing for multithreading. To lower the implementation cost, a GPU performance model is developed to detect the bottlenecks and provide design direction for the GPU kernel. Based on these optimization techniques, a prototype implementation of DPI at 150 Gb/s is achieved on a single NVIDIA K20 GPU.

Near Video-Rate Optical Coherence Elastography by Acceleration With a Graphics Processing Unit

We present a graphics processing unit (GPU)-accelerated optical coherence elastography (OCE) system capable of generating strain images (elastograms) of soft tissue at near video-rates. The system implements phase-sensitive compression OCE using a pipeline of GPU kernel functions to enable a highly parallel implementation of OCE processing using the OpenCL framework. Developed on a commercial-grade GPU and desktop computer, the system achieves a processing rate of 21 elastograms per second at an image size of 960 × 400 pixels, enabling high-rate visualization during acquisition. The system is demonstrated on both tissue-simulating phantoms and fresh ex vivo mouse muscle. To the best of our knowledge, this is the first implementation of near video-rate OCE and the fastest reported OCE processing rate, enabling, for the first time, a system capable of computing and displaying OCE elastograms interactively during acquisition. This advance provides new opportunities for medical imaging of soft tissue stiffness using optical methods.

A fast GPU-based Monte Carlo simulation of proton transport with detailed modeling of nonelastic interactions.

Very fast Monte Carlo (MC) simulations of proton transport have been implemented recently on graphics processing units (GPUs). However, these MCs usually use simplified models for nonelastic proton-nucleus interactions. Our primary goal is to build a GPU-based proton transport MC with detailed modeling of elastic and nonelastic proton-nucleus collisions. Using the cuda framework, the authors implemented GPU kernels for the following tasks: (1) simulation of beam spots from our possible scanning nozzle configurations, (2) proton propagation through CT geometry, taking into account nuclear elastic scattering, multiple scattering, and energy loss straggling, (3) modeling of the intranuclear cascade stage of nonelastic interactions when they occur, (4) simulation of nuclear evaporation, and (5) statistical error estimates on the dose. To validate our MC, the authors performed (1) secondary particle yield calculations in proton collisions with therapeutically relevant nuclei, (2) dose calculations in homogeneous phantoms, (3) recalculations of complex head and neck treatment plans from a commercially available treatment planning system, and compared with (GEANT)4.9.6p2/TOPAS. Yields, energy, and angular distributions of secondaries from nonelastic collisions on various nuclei are in good agreement with the (GEANT)4.9.6p2 Bertini and Binary cascade models. The 3D-gamma pass rate at 2%-2 mm for treatment plan simulations is typically 98%. The net computational time on a NVIDIA GTX680 card, including all CPU-GPU data transfers, is ∼ 20 s for 1 × 10(7) proton histories. Our GPU-based MC is the first of its kind to include a detailed nuclear model to handle nonelastic interactions of protons with any nucleus. Dosimetric calculations are in very good agreement with (GEANT)4.9.6p2/TOPAS. Our MC is being integrated into a framework to perform fast routine clinical QA of pencil-beam based treatment plans, and is being used as the dose calculation engine in a clinically applicable MC-based IMPT treatment planning system. The detailed nuclear modeling will allow us to perform very fast linear energy transfer and neutron dose estimates on the GPU.

GPU acceleration of Volumetric Lattice Boltzmann Method for patient-specific computational hemodynamics

Volumetric Lattice Boltzmann Method (VLBM) has been recently developed for solving complex flow with arbitrary curved boundaries. The VLBM regards fluid particles are uniformly distributed in cells and distinguishes fluid, solid, and boundary cells by introducing a volumetric parameter P(x,t) defining the percentage of solid volume in each cell. The advantages of VLBM stem from the self-regulation of P(x,t) in the volumetric lattice Boltzmann equation (VLBE) for particle collision and streaming with no spatial interpolation when dealing with an arbitrarily curved boundary with or without motion. First, the VLBE satisfies mass conservation strictly. Second, the implementation of VLBM is rather simple after the solid volume percentages are determined in boundary cells. And third, no-slip boundary condition is integrated in the streaming formulation thus significantly enhances the capability of parallelization. In this paper, we perform GPU (Graphics Processing Unit) parallelization for VLBM using a uniform computing scheme for both fluid and boundary cells. In contrast to the traditional LBM acceleration, the boundary conditions have to be imposed over boundary nodes, where branching operations are required to identify boundary nodes from others, the VLBM implementation does not need to distinguish fluid and boundary cells in the computation so that branching is minimized and the GPU kernel execution is accelerated. Furthermore, the algorithmic steps are optimized to improve coalesced access of GPU memory and avoid race condition. An application study is on a pulsatile blood flow in a patient-specific carotid artery segmented from an anonymous clinical CT image and more than 30 times speedup over the serial counterpart. Simulations of fluid dynamics and wall shear stress (WSS) are presented and known velocity skewness and WSS distributions are captured. The GPU accelerated VLBM is promising to perform patient-specific computational hemodynamics within clinical accepted time frame and is expected to reveal quantitative real-time blood flow in living human arteries to aid clinical assessment of cardiovascular diseases.