Graphics Processing Units Performance Research Articles

New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs

On-chip caches of graphics processing units (GPUs) have contributed to improved GPU performance by reducing long memory access latency. However, cache efficiency remains low despite the facts that recent GPUs have considerably mitigated the bottleneck problem of L1 data cache. Although the cache miss rate is a reasonable metric for cache efficiency, it is not necessarily proportional to GPU performance. In this study, we introduce a second key determinant to overcome the problem of predicting the performance gains from L1 data cache based on the assumption that miss rate only is not accurate. The proposed technique estimates the benefits of the cache by measuring the balance between cache efficiency and throughput. The throughput of the cache is predicted based on the warp occupancy information in the warp pool. Then, the warp occupancy is used for a second bypass phase when workloads show an ambiguous miss rate. In our proposed architecture, the L1 data cache is turned off for a long period when the warp occupancy is not high. Our two-level bypassing technique can be applied to recent GPU models and improves the performance by 6% on average compared to the architecture without bypassing. Moreover, it outperforms the conventional bottleneck-based bypassing techniques.

Ray tracing collision detection based on GPU pipeline reorganization

This paper proposes a parallel collision detection algorithm based on ray tracing to improve GPU performance. The algorithm uses parallel ray tracing pipeline technology to improve the performance of the graphics processing unit (GPU). When the GPU combines multiple ray tracing, it will cause the work group threads to diverge, resulting in too many idle threads, which will reduce the performance of the system. In order to avoid thread divergence, a new collision detection technology for the detailed detection stage of ray tracing using a pipeline structure is proposed. By optimizing the addition of buffers, the ray tracing is divided into three stages, and the pipeline is designed to integrate multiple rays tracking to avoid unnecessary synchronization issues between the CPU and GPU, thereby improving system performance. The comparison experiment results show that the proposed algorithm improves the system performance by 2.7 times.

The FastEddy® Resident‐GPU Accelerated Large‐Eddy Simulation Framework: Model Formulation, Dynamical‐Core Validation and Performance Benchmarks

AbstractThis paper introduces a new large‐eddy simulation model, FastEddy®, purpose built for leveraging the accelerated and more power‐efficient computing capacity of graphics processing units (GPUs) toward adopting microscale turbulence‐resolving atmospheric boundary layer simulations into future numerical weather prediction activities. Here a basis for future endeavors with the FastEddy® model is provided by describing the model dry dynamics formulation and investigating several validation scenarios that establish a baseline of model predictive skill for canonical neutral, convective, and stable boundary layer regimes, along with boundary layer flow over heterogeneous terrain. The current FastEddy® GPU performance and efficiency gains versus similarly formulated, state‐of‐the‐art CPU‐based models is determined through scaling tests as 1 GPU to 256 CPU cores. At this ratio of GPUs to CPU cores, FastEddy® achieves 6 times faster prediction rate than commensurate CPU models under equivalent power consumption. Alternatively, FastEddy® uses 8 times less power at this ratio under equivalent CPU/GPU prediction rate. The accelerated performance and efficiency gains of the FastEddy® model permit more broad application of large‐eddy simulation to emerging atmospheric boundary layer research topics through substantial reduction of computational resource requirements and increase in model prediction rate.

Boosting Free-Energy Perturbation Calculations with GPU-Accelerated NAMD.

Harnessing the power of graphics processing units (GPUs) to accelerate molecular dynamics (MD) simulations in the context of free-energy calculations has been a longstanding effort toward the development of versatile, high-performance MD engines. We report a new GPU-based implementation in NAMD of free-energy perturbation (FEP), one of the oldest, most popular importance-sampling approaches for the determination of free-energy differences that underlie alchemical transformations. Compared to the CPU implementation available since 2001 in NAMD, our benchmarks indicate that the new implementation of FEP in traditional GPU code is about four times faster, without any noticeable loss of accuracy, thereby paving the way toward more affordable free-energy calculations on large biological objects. Moreover, we have extended this new FEP implementation to a code path highly optimized for a single-GPU node, which proves to be up to nearly 30 times faster than the CPU implementation. Through optimized GPU performance, the present developments provide the community with a cost-effective solution for conducting FEP calculations. The new FEP-enabled code has been released with NAMD 3.0.

Efficient Performance Estimation and Work-Group Size Pruning for OpenCL Kernels on GPUs

Graphic Processing Units (GPUs) play a vital role in state-of-the-art high-performance scientific computing realm and research work towards its performance analysis is crucial but nontrivial. Extant GPU performance models are far from practical use, while fine-grained GPU simulation requires a considerably large time cost. Moreover, massive amounts of designs with various program inputs and parameter settings pose a challenge for efficient performance estimation and tuning of parallel GPU applications. To this end, this article presents a hybrid framework for the efficient performance estimation and work-group size pruning of OpenCL workloads on GPUs. The framework contains a static module used to extract the kernel execution trace from the high-level source code and a dynamical module used to mimic the kernel execution flow to estimate the runtime performance. For the design space pruning, an extra analysis is performed to filter out the redundant work-group sizes with duplicated execution traces and inferior pipelines. The proposed framework does not require any program runs to estimate the performance and find the optimal or near-optimal designs. Experiments on four Commercial Off-The-Shelf (COTS) Nvidia GPUs show that the framework can predict the runtime performance with an average error of 17.04 percent and reduce the program design space by an average of 78.47 percent.

Communication protocol optimization for enhanced GPU performance

The U.S. Department of Energy CORAL program systems SUMMIT and SIERRA are based on hybrid servers comprising IBM POWER9 CPUs and NVIDIA V100 graphics processing units (GPUs) connected by two extended data rate (EDR) links to a high-speed InfiniBand Network. A major challenge to the communication software stack is to optimize performance for all combinations of data origin and destination: host or GPU memory, same or different server. Alternate paths exist for routing data from GPU memory. When origin and destination are on different servers, it can be sent either via host memory or bypassing host memory with GPU direct feature. When origin and destination are on the same server, host memory can be bypassed with peer-to-peer inter process communication (IPC). For large messages pipelining makes host memory data path competitive with GPU direct. In this article, we explain the techniques used in Spectrum MPI Parallel Active Message Interface layer to cache memory types and attributes in order to reduce the overhead associated with calling the CUDA application programming interface (API); in addition, we detail the different protocols used for different memory types, device memory, managed memory, and host memory. To illustrate, the caching technique achieved a device-to-device latency improvement of 26% for intranode transfers and 19% for internode transfers.

GPU accelerated study of a dual-frequency driven single bubble in a 6-dimensional parameter space: The active cavitation threshold.

The active cavitation threshold of a dual-frequency driven single spherical gas bubble is studied numerically. This threshold is defined as the minimum intensity required to generate a given relative expansion (Rmax-RE)/RE, where RE is the equilibrium size of the bubble and Rmax is the maximum bubble radius during its oscillation. The model employed is the Keller-Miksis equation that is a second order ordinary differential equation. The parameter space investigated is composed by the pressure amplitudes, excitation frequencies, phase shift between the two harmonic components and by the equilibrium bubble radius (bubble size). Due to the large 6-dimensional parameter space, the number of the parameter combinations investigated is approximately two billion. Therefore, the high performance of graphics processing units is exploited; our in-house code is written in C++ and CUDA C software environments. The results show that for (Rmax-RE)/RE=2, the best choice of the frequency pairs depends on the bubble size. For small bubbles, below 3μm, the best option is to use just a single frequency of a low value in the giant response region. For medium sized bubbles, between 3μm and 6μm, the optimal choice is the mixture of low frequency (giant response) and main resonance frequency. For large bubbles, above 6μm, the main resonance dominates the active cavitation threshold. Increasing the prescribed relative expansion value to (Rmax-RE)/RE=3, the optimal choice is always single frequency driving with the lowest value (20kHz here). Thus, in this case, the giant response always dominates the active cavitation threshold. The phase shift between the harmonic components of the dual-frequency driving (different frequency values) has no effect on the threshold.

Multi-GPU Island-Based Genetic Algorithm

Genetic algorithms are effective in solving many optimization tasks. However, the long execution time associated with it prevents its use in many domains. In this paper, we propose a new approach for parallel implementation of genetic algorithm on graphics processing units (GPUs) using CUDA programming model. This paper introduces a novel implementation of the genetic algorithm exploiting a multi-GPU cluster. The proposed implementation employs an island-based genetic algorithm where every GPU evolves a single island. Recently there has been a growing interest in developing parallel algorithms using graphic processing units (GPU) also referred as GPU computation. Advances in the video gaming industry have led to the production of low-cost, high performance graphics processing units that possess more memory bandwidth and computational capability than central processing units (CPUs). As GPUs are available in personal computers, and they are easy to use and manage through several GPU programming languages, graphics engines are being adopted widely in scientific computing.

Efficient ResNet Model to Predict Protein-Protein Interactions With GPU Computing

Protein-protein interactions (PPI) play an important role in the cell activities of organisms. The deep research about PPI can help humans understand the mechanism of life activities and apply protein functions better. Nowadays, PPI prediction algorithms based on amino acid sequences using the recurrent neural network (RNN) can overcome the disadvantages of traditional biological experimental methods and achieve high accuracy. However, these algorithms are usually time-consuming and cannot take full advantage of graphics processing units (GPU) with efficient computation performance to accelerate PPI prediction, because the RNN model considers the time series of sequences. In this paper, we propose an efficient algorithm based on the residual network (ResNet) model to predict PPI (ResPPI). Our algorithm uses the embedding method to represent amino acid sequences, combining the advantages of powerful feature extraction capabilities of the ResNet with deep layers and GPU performance. The experimental results show that the ResPPI algorithm can ensure high accuracy and reduce training time greatly. Based on the ordinary GPU device, compared with the state-of-the-art LSTM model, the speed of the ResPPI algorithm is five times faster than that of the LSTM, whereas the ResPPI algorithm can achieve similar accuracy to the LSTM. Besides, in the case of unbalanced datasets, the ResPPI algorithm can perform better.

Improving GPU Performance with a Power-Aware Streaming Multiprocessor Allocation Methodology

Graphics processing units (GPUs) are extensively used as accelerators across multiple application domains, ranging from general purpose applications to neural networks, and cryptocurrency mining. The initial utilization paradigm for GPUs was one application accessing all the resources of the GPU. In recent years, time sharing is broadly used among applications of a GPU, nevertheless, spatial sharing is not fully explored. When concurrent applications share the computational resources of a GPU, performance can be improved by eliminating idle resources. Additionally, the incorporation of GPUs in embedded and mobile devices increases the demand for power efficient computation due to battery limitations. In this article, we present an allocation methodology for streaming multiprocessors (SMs). The presented methodology works for two concurrent applications on a GPU and determines an allocation scheme that will provide power efficient application execution, combined with improved GPU performance. Experimental results show that the developed methodology yields higher throughput while achieving improved power efficiency, compared to other SM power-aware and performance-aware policies. If the presented methodology is adopted, it will lead to higher performance of applications that are concurrently executing on a GPU. This will lead to a faster and more efficient acceleration of execution, even for devices with restrained energy sources.

Highly Concurrent Latency-tolerant Register Files for GPUs

Graphics Processing Units (GPUs) employ large register files to accommodate all active threads and accelerate context switching. Unfortunately, register files are a scalability bottleneck for future GPUs due to long access latency, high power consumption, and large silicon area provisioning. Prior work proposes hierarchical register file to reduce the register file power consumption by caching registers in a smaller register file cache. Unfortunately, this approach does not improve register access latency due to the low hit rate in the register file cache. In this article, we propose the Latency-Tolerant Register File (LTRF) architecture to achieve low latency in a two-level hierarchical structure while keeping power consumption low. We observe that compile-time interval analysis enables us to divide GPU program execution into intervals with an accurate estimate of a warp’s aggregate register working-set within each interval. The key idea of LTRF is to prefetch the estimated register working-set from the main register file to the register file cache under software control, at the beginning of each interval, and overlap the prefetch latency with the execution of other warps. We observe that register bank conflicts while prefetching the registers could greatly reduce the effectiveness of LTRF. Therefore, we devise a compile-time register renumbering technique to reduce the likelihood of register bank conflicts. Our experimental results show that LTRF enables high-capacity yet long-latency main GPU register files, paving the way for various optimizations. As an example optimization, we implement the main register file with emerging high-density high-latency memory technologies, enabling 8× larger capacity and improving overall GPU performance by 34%.

A novel warp scheduling scheme considering long-latency operations for high-performance GPUs

Graphics processing units (GPUs) have become one of the best platforms for exploiting the plentiful thread-level parallelism of applications. However, GPUs continue to underutilize their hardware resources for optimizing the performance of numerous general-purpose applications. One primary reason for this is the inefficiency of existing warp schedulers in hiding long-latency operations such as global loads and stores. This study proposes a long-latency operation-based warp scheduler to improve GPU performance. In the proposed warp scheduler, warps are partitioned into different pools based on the characteristics of instructions that are subsequently executed. Specifically, this warp scheduler uses warps that are likely waiting for long-latency operations for a guiding role. Meanwhile, other warps perform filling roles (i.e., to overlap the latencies caused by the guiding warps). Our experimental results demonstrate that the proposed warp scheduler improves GPU performance by 24.4% on average as compared to the conventional warp scheduler.

Ballooning Graphics Memory Space in Full GPU Virtualization Environments

Advances in virtualization technology have enabled multiple virtual machines (VMs) to share resources in a physical machine (PM). With the widespread use of graphics-intensive applications, such as two-dimensional (2D) or 3D rendering, many graphics processing unit (GPU) virtualization solutions have been proposed to provide high-performance GPU services in a virtualized environment. Although elasticity is one of the major benefits in this environment, the allocation of GPU memory is still static in the sense that after the GPU memory is allocated to a VM, it is not possible to change the memory size at runtime. This causes underutilization of GPU memory or performance degradation of a GPU application due to the lack of GPU memory when an application requires a large amount of GPU memory. In this paper, we propose a GPU memory ballooning solution calledgBalloonthat dynamically adjusts the GPU memory size at runtime according to the GPU memory requirement of each VM and the GPU memory sharing overhead. ThegBalloonextends the GPU memory size of a VM by detecting performance degradation due to the lack of GPU memory. ThegBalloonalso reduces the GPU memory size when the overcommitted or underutilized GPU memory of a VM creates additional overhead for the GPU context switch or the CPU load due to GPU memory sharing among the VMs. We implemented thegBalloonby modifying thegVirt, a full GPU virtualization solution for Intel’s integrated GPUs. Benchmarking results show that thegBalloondynamically adjusts the GPU memory size at runtime, which improves the performance by up to 8% against thegVirtwith 384 MB of high global graphics memory and 32% against thegVirtwith 1024 MB of high global graphics memory.

Performance Analysis and Optimization of Graphics Processing Unit

With the advancement of technology, we require high performance computation for large applications. A Graphics Processing Unit (GPU) provides a platform where huge computation for large data using parallel processing and data analysis is performed. GPU architectures are becoming more and more programmable, and providing dramatic speedups for a number of general-purpose applications. This paper uses NVIDIA’s CUDA language to demostrate the capabilities of GPUs for large applications, and explores some specific coding that improves the performance on the GPU. GPU performance has been compared with CPU architecture.

ITAP

Graphics Processing Units (GPUs) are widely used as the accelerator of choice for applications with massively data-parallel tasks. However, recent studies show that GPUs suffer heavily from resource underutilization, which, combined with their large static power consumption, imposes a significant power overhead. One of the most power-hungry components of a GPU—the execution units—frequently experience idleness when (1) an underutilized warp is issued to the execution units, leading to partial lane idleness, and (2) there is no active warp to be issued for the execution due to warp stalls (e.g., waiting for memory access and synchronization). Although large in total, the idle time of execution units actually comes from short but frequent stalls, leaving little potential for common power saving techniques, such as power-gating. In this article, we propose ITAP , a novel idle-time-aware power management technique, which aims to effectively reduce the static energy consumption of GPU execution units. By taking advantage of different power management techniques (i.e., power-gating and different levels of voltage scaling), ITAP employs three static power reduction modes with different overheads and capabilities of static power reduction. ITAP estimates the idle period length of execution units using prediction and peek-ahead techniques in a synergistic way and then applies the most appropriate static power reduction mode based on the estimated idle period length. We design ITAP to be power-aggressive or performance-aggressive, not both at the same time. Our experimental results on several workloads show that the power-aggressive design of ITAP outperforms the state-of-the-art solution by an average of 27.6% in terms of static energy savings, with less than 2.1% performance overhead. However, the performance-aggressive design of ITAP improves the static energy savings by an average of 16.9%, while keeping the GPU performance almost unaffected (i.e., up to 0.4% performance overhead) compared to the state-of-the-art static energy savings mechanism.

A Practically Applicable Performance Prediction Model Based on Capabilities of Texture Mapping Units for Mobile GPUs

The power consumption models of mobile application processors have emerged as key objects of interest following the tremendous growth in mobile device production given that such consumption is an important factor in the graphics performance of mobile technologies. Conventionally, the performance of the graphics processing units (GPUs) depends critically on texture mapping units, which is why the number of such GPU components and texture fill rates value prominently whenever the GPU performance is evaluated. Our previous work has established a model to predict maximum performance based on unified shaders. By extending the work, this paper developed a practically applicable GPU performance prediction model on the basis of texture mapping performance. The effects of increased texture mapping units on unified shader performance and GPU efficiency were examined, and a performance prediction model based on the number of frames per second (FPS) was constructed. For these purposes, a benchmark related to texture mapping units was formulated and the experiments were conducted to determine utilization factors that are relevant to GPU performance and efficiency. The final stage in model construction involved establishing a relationship between the previously investigated utilization factors and relevant resources that are consumed during graphics processing. The experimental results showed that the proposed prediction model produced an average error rate of 5.77%.

GPU Energy Consumption Optimization With a Global-Based Neural Network Method

With the widespread use of smart technologies, graphics processing unit (GPU) power-optimization issues are becoming increasingly important. Many researchers have tried to use dynamic voltage and frequency scaling (DVFS) technology to optimize a GPU’s internal energy consumption. However, DVFS energy management often has difficulty balancing GPU performance and energy efficiency. This paper aims to implement a DVFS energy management strategy. We constructed a new type of neural network to a GPU-based energy management scheme, implemented the global-based DVFS model, and explored its implementation details. Using a master-slave model, we built a global energy control solution strategy. This strategy performs global collaborative DVFS adjustments on the GPU’s energy consumption module based on task characteristics. Through the software construction and implementation of the global-based DVFS model, we proved that the strategy improves the GPU performance while improving the GPU’s energy efficiency. We conducted performance and energy tests on three GPUs on the Tesla, Fermi, and Kepler platforms. The experiments showed that this strategy improved the performance and power consumption of GPUs based on each of the platforms.

Real-time parallel 3D multiple particle tracking with single molecule centrifugal force microscopy.

Real-time tracking of multiple particles is key for quantitative analysis of dynamic biophysical processes and materials science via time-lapse microscopy image data, especially for single molecule biophysical techniques, such as magnetic tweezers and centrifugal force microscopy. However, real-time multiple particle tracking with high resolution is limited by the current imaging processes or tracking algorithms. Here, we demonstrate 1 nm resolution in three dimensions in real-time with a graphics-processing unit (GPU) based on a compute unified device architecture (CUDA) parallel computing framework instead of only a central processing unit (CPU). We also explore the trade-offs between processing speed and size of the utilized regions of interest and a maximum speedup of 137 is achieved with the GPU compared with the CPU. Moreover, we utilize this method with our recently self-built centrifugal force microscope (CFM) in experiments that track multiple DNA-tethered particles. Our approach paves the way for high-throughput single molecule techniques with high resolution and efficiency. LAY DESCRIPTION: Particles are widely used as probes in life sciences through their motions. In single molecule techniques such as optical tweezers and magnetic tweezers, microbeads are used to study intermolecular or intramolecular interactions via beads tracking. Also tracking multiple beads' motions could study cell-cell or cell-ECM interactions in traction force microscopy. Therefore, particle tracking is of key important during these researches. However, parallel 3D multiple particle tracking in real-time with high resolution is a challenge either due to the algorithm or the program. Here, we combine the performance of CPU and CUDA-based GPU to make a hybrid implementation for particle tracking. In this way, a speedup of 137 is obtained compared the program before only with CPU without loss of accuracy. Moreover, we improve and build a new centrifugal force microscope for multiple single molecule force spectroscopy research in parallel. Then we employed our program into centrifugal force microscope for DNA stretching study. Our results not only demonstrate the application of this program in single molecule techniques, also indicate the capability of multiple single molecule study with centrifugal force microscopy.

Performance Evaluation of SW Algorithm on NVIDIA GeForce GTX TITAN X Graphic Processing Unit (GPU)

Nowadays, the requirement for high performance and sensitive alignment tools have increased after the advantage of the Deoxyribonucleic Acid (DNA) and molecular biology has been figured out through Bioinformatics study. Therefore, this paper reports the performance evaluation of parallel Smith-Waterman Algorithm implementation on the new NVIDIA GeForce GTX Titan X Graphic Processing Unit (GPU) compared to the Central Processing Unit (CPU) running on Intel® CoreTM i5-4440S CPU 2.80GHz. Both of the design were developed using C-programming language and targeted to the respective platform. The code for GPU was developed and compiled using NVIDIA Compute Unified Device Architecture (CUDA). It clearly recorded that, the performance of GPU based computational is better compared to the CPU based. These results indicate that the GPU based DNA sequence alignment has a better speed in accelerating the computational process of DNA sequence alignment.

GPU acceleration and performance of the particle-beam-dynamics code Elegant

Elegant is an accelerator physics and particle-beam dynamics code widely used for modeling and design of a variety of high-energy particle accelerators and accelerator-based systems. In this paper we discuss a recently developed version of the code that can take advantage of CUDA-enabled graphics processing units (GPUs) to achieve significantly improved performance for a large class of simulations that are important in practice. The GPU version is largely defined by a framework that simplifies implementations of the fundamental kernel types that are used by Elegant: particle operations, reductions, particle loss, histograms, array convolutions and random number generation. Accelerated performance on the Titan Cray XK-7 supercomputer is approximately 6–10 times better with the GPU than all the CPU cores associated with the same node count. In addition to performance, the maintainability of the GPU-accelerated version of the code was considered a key design objective. Accuracy with respect to the CPU implementation is also a core consideration. Four different methods are used to ensure that the accelerated code faithfully reproduces the CPU results. Program summaryProgram Title: Kernels from the GPU-accelerated ElegantProgram Files doi:http://dx.doi.org/10.17632/jc465zy7p5.1Licensing provisions: MITProgramming language: C/C++/CUDANature of problem: The original design of the Elegant accelerator physics code was implemented on central processing units with message-passing interface parallelization. This implementation is not able to use next-generation multicore systems.Solution method: In this package we develop routines based on the CUDA language extensions to C++ that enable porting the Elegant code to be run on graphics processing units (GPUs). Special consideration is given to algorithms that require collective communication on the GPU.Additional comments including restrictions and unusual features: The full Elegant source code is freely available from Argonne National Laboratory and these distributions include the GPU code in the later releases.