Graphics Processing Units Performance Research Articles

SHREG: Mitigating register redundancy in GPUs

Graphics Processing Units (GPUs) have become dominant accelerators for Machine Learning (ML) and High-Performance Computing (HPC) applications due to their massive parallelism capabilities, through the utilization of general matrix-to-matrix multiplication (GEMM) kernels. However, GEMM kernels often suffer from duplicated memory requests, mainly caused by matrix tiling used for handling large matrices. While GPUs have adopted programmable shared memory to mitigate this issue by preserving frequently reused data in shared memory, GEMM still introduces duplication in register files. Our observations show that the matrix tiling issues memory requests to the same shared memory address for neighboring threads, and this results in a substantial increase in the number of duplicated data in the register files. Such duplication degrades GPU performance by limiting warp-level parallelism due to the register shortage and redundant memory requests to shared memory. We find that the data duplication can be categorized into two types that occur with fixed patterns during the matrix tiling. Based on these observations, we introduce SHREG, an architecture design that enables different threads to share registers for overlapped data from shared memory, effectively reducing duplicated data within the register files. By leveraging the duplication patterns, SHREG utilizes register sharing and improves performance with minimal hardware overhead. Our evaluation shows that SHREG improves performance by 31.4% on various ML applications over the baseline GPU.

Can GPU performance increase faster than the code error rate?

Graphics processing units (GPUs) are the reference architecture to accelerate high-performance computing applications and the training/interference of convolutional neural networks. For both these domains, performance and reliability are two of the main constraints. It is believed that the only way to increase reliability is to sacrifice performance, e.g., using redundancies. We show in this paper that this is not always the case. As a very promising result, we found that most GPUs performance improvements also bring the benefit of increasing the number of executions correctly completed before experiencing a silent data corruption (SDC). We consider four different common GPUs’ performance optimizations: architectural solutions, software implementations, compiler optimizations, and threads degree of parallelism. We compare different implementations of a variety of parallel codes and, through beam experiments and applications profiling, we show that the performance improvement typically (but not necessarily) increases the GPU SDC rate. Nevertheless, for the vast majority of the configurations the performance gain is much higher than the SDC rate increase, allowing to process a higher amount of correct data. As we show, the programmer choices can increase up to 25×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$25\ imes {}$$\\end{document} the number of correctly completed executions without redesigning the algorithm nor including specific hardening solutions.

Enhancing GPU performance and energy efficiency: Innovative strategies for sustainable computing

With the rapid advancement of science and information technology, Graphics Processing Units (GPUs) have become indispensable tools in contemporary scientific research and industrial production. This paper delves into optimizing GPU performance and energy efficiency, with a specific focus on five aspects: core count and layout, memory hierarchy and cache design, clock speed, specialized hardware units for specific tasks, and interconnections among GPU components. Increasing core count generally enhances GPU performance but also elevates energy consumption. Optimizing core count and layout strikes a balance between performance and energy use. Memory hierarchy and cache design are crucial for handling the inherent parallelism of GPU architecture; optimizing these aspects boosts GPU efficiency and sustainability. Clock speed is a vital performance indicator, with the right speed achieving an optimal balance between performance and energy use. Tailoring hardware units for specific tasks enhances computational efficiency and lowers energy consumption. Optimizing interconnections between GPU components enhances data transfer efficiency, yielding higher performance. Through comprehensive research and optimization in these five areas, this paper introduces innovative strategies and techniques to boost GPU performance and reduce energy consumption, laying the foundation for sustainable computing development.

Accelerating atmospheric physics parameterizations using graphics processing units

As part of a project aimed at exploring the use of next-generation high-performance computing technologies for numerical weather prediction, we have ported two physics modules from the Common Community Physics Package (CCPP) to Graphics Processing Unit (GPU) and obtained accelerations of up to 10× relative to a comparable multi-core CPU. The physics parameterizations accelerated in this work are the aerosol-aware Thompson microphysics (TH) scheme and the Grell–Freitas (GF) cumulus convection scheme. Microphysics schemes are among the most time-consuming physics parameterizations, second to only radiative process schemes, and our results show better acceleration for the TH scheme than the GF scheme. Multi-GPU implementations of the schemes show acceptable weak scaling in a single node with 8 GPUs, and perfect weak scaling on multiple nodes using one GPU per node. The lack of inter-node communication for column physics parameterizations contributes to their scalability, however, physics parameterizations are run along with dynamics, so the overall multi-GPU performance is often governed by the latter. In the context of optimizing CCPP physics modules, our observations underscore that the extensive use of automatic arrays within inner subroutines hampers GPU performance due to serialized memory allocations. We have used the OpenACC directive programming language for this work because it allows for easy porting of large amounts of code and makes code maintenance more manageable compared to low-level languages like CUDA and OpenCL.

Analyzing GPU Performance in Virtualized Environments: A Case Study

The graphics processing unit (GPU) plays a crucial role in boosting application performance and enhancing computational tasks. Thanks to its parallel architecture and energy efficiency, the GPU has become essential in many computing scenarios. On the other hand, the advent of GPU virtualization has been a significant breakthrough, as it provides scalable and adaptable GPU resources for virtual machines. However, this technology faces challenges in debugging and analyzing the performance of GPU-accelerated applications. Most current performance tools do not support virtual GPUs (vGPUs), highlighting the need for more advanced tools. Thus, this article introduces a novel performance analysis tool that is designed for systems using vGPUs. Our tool is compatible with the Intel GVT-g virtualization solution, although its underlying principles can apply to many vGPU-based systems. Our tool uses software tracing techniques to gather detailed runtime data and generate relevant performance metrics. It also offers many synchronized graphical views, which gives practitioners deep insights into GVT-g operations and helps them identify potential performance bottlenecks in vGPU-enabled virtual machines.

Fast grid to grid interpolation for radio interferometric imaging

Backward and forward interpolations on a Fourier grid are computationally expensive operations for radio interferometric imaging algorithms. By merging these operations, we propose the Grid to Grid (G2G) method which aims to reduce the computational cost and memory footprint. We have also shown that the oversampling factor used for the convolution function in the G2G method strongly impacted the accuracy and computational speed. Acceleration on graphics processing units (GPU), well suited for this embarrassingly parallel algorithm, has been studied mainly for backward operation. Thus, we propose a GPU and CPU implementation of the G2G method on Nvidia A100 and Intel Ice Lake processors. Experiments have shown a GPU performance improvement with a Fourier-point throughput better than up to 37% regarding standard gridder and degridder.

The Design of an Intelligent Lightweight Stock Trading System Using Deep Learning Models: Employing Technical Analysis Methods

Individual investors often struggle to predict stock prices due to the limitations imposed by the computational capacities of personal laptop Graphics Processing Units (GPUs) when running intensive deep learning models. This study proposes solving these GPU constraints by integrating deep learning models with technical analysis methods. This integration significantly reduces analysis time and equips individual investors with the ability to identify stocks that may yield potential gains or losses in an efficient manner. Thus, a comprehensive buy and sell algorithm, compatible with average laptop GPU performance, is introduced in this study. This algorithm offers a lightweight analysis method that emphasizes factors identified by technical analysis methods, thereby providing a more accessible and efficient approach for individual investors. To evaluate the efficacy of this approach, we assessed the performance of eight deep learning models: long short-term memory (LSTM), a convolutional neural network (CNN), bidirectional LSTM (BiLSTM), CNN Attention, a bidirectional gated recurrent unit (BiGRU) CNN BiLSTM Attention, BiLSTM Attention CNN, CNN BiLSTM Attention, and CNN Attention BiLSTM. These models were used to predict stock prices for Samsung Electronics and Celltrion Healthcare. The CNN Attention BiLSTM model displayed superior performance among these models, with the lowest validation mean absolute error value. In addition, an experiment was conducted using WandB Sweep to determine the optimal hyperparameters for four individual hybrid models. These optimal parameters were then implemented in each model to validate their back-testing rate of return. The CNN Attention BiLSTM hybrid model emerged as the highest-performing model, achieving an approximate rate of return of 5 percent. Overall, this study offers valuable insights into the performance of various deep learning and hybrid models in predicting stock prices. These findings can assist individual investors in selecting appropriate models that align with their investment strategies, thereby increasing their likelihood of success in the stock market.

Adopting GPU computing to support DL-based Earth science applications

ABSTRACT With the advancement of Artificial Intelligence (AI) technologies and accumulation of big Earth data, Deep Learning (DL) has become an important method to discover patterns and understand Earth science processes in the past several years. While successful in many Earth science areas, AI/DL applications are often challenging for computing devices. In recent years, Graphics Processing Unit (GPU) devices have been leveraged to speed up AI/DL applications, yet computational performance still poses a major barrier for DL-based Earth science applications. To address these computational challenges, we selected five existing sample Earth science AI applications, revised the DL-based models/algorithms, and tested the performance of multiple GPU computing platforms to support the applications. Application software packages, performance comparisons across different platforms, along with other results, are summarized. This article can help understand how various AI/ML Earth science applications can be supported by GPU computing and help researchers in the Earth science domain better adopt GPU computing (such as supermicro, GPU clusters, and cloud computing-based) for their AI/ML applications, and to optimize their science applications to better leverage the computing device.

A novel GPU based Geo-Location Inference Attack on WebGL framework

In the past few years, graphics processing units (GPUs) have become an indispensable part of modern computer systems, not only for graphics rendering but also for intensive parallel computing. Given that many tasks running on GPUs contain sensitive information, security concerns have been raised, especially about potential GPU information leakage. Previous works have shown such concerns by showing that attackers can use GPU memory allocations or performance counters to measure victim side effects. However, such an attack has a critical drawback that it requires a victim to install desktop applications or mobile apps yielding it uneasy to be deployed in the real world. In this paper, we solve this drawback by proposing a novel GPU-based side-channel Geo-Privacy inference attack on the WebGL framework, namely, GLINT (stands for Geo-Location Inference Attack). GLINT merely utilizes a lightweight browser extension to measure the time elapsed to render a sequence of frames on well-known map websites, e.g., Google Maps, or Baidu Maps. The measured stream of time series is then employed to infer geologically privacy-sensitive information, such as a search on a specific location. Upon retrieving the stream, we propose a novel online segmentation algorithm for streaming data to determine the start and end points of privacy-sensitive time series. We then combine the DTW algorithm and KNN algorithm on these series to conclude the final inference on a user’s geo-location privacy.We conducted real-world experiments to testify our attack. The experiments show that GeoInfer can correctly infer more than 83% of user searches regardless of the locations and map websites, meaning that our Geo-Privacy inference attack is accurate, practical, and robust. To counter this attack, we implemented a defense strategy based on Differential Privacy to hinder obtaining accurate rendering data. We found that this defense mechanism managed to reduce the average accuracy of the attack model by more than 70%, indicating that the attack was no longer effective. We have fully implemented GLINT and open-sourced it for future follow-up research.

Offloading electromagnetic shower transport to GPUs

Making general particle transport simulation for high-energy physics (HEP) single-instruction-multiple-thread (SIMT) friendly, to take advantage of accelerator hardware, is an important alternative for boosting the throughput of simulation applications. To date, this challenge is not yet resolved, due to difficulties in mapping the complexity of Geant4 components and workflow to the massive parallelism features exposed by graphics processing units (GPU). The AdePT project is one of the R&D initiatives tackling this limitation and exploring GPUs as potential accelerators for offloading some part of the CPU simulation workload. Our main target is to implement a complete electromagnetic shower demonstrator working on the GPU. The project is the first to create a full prototype of a realistic electron, positron, and gamma electromagnetic shower simulation on GPU, implemented as either a standalone application or as an extension of the standard Geant4 CPU workflow. Our prototype currently provides a platform to explore many optimisations and different approaches. We present the most recent results and initial conclusions of our work, using both a standalone GPU performance analysis and a first implementation of a hybrid workflow based on Geant4 on the CPU and AdePT on the GPU.

OSM: Off-Chip Shared Memory for GPUs

Graphics Processing Units (GPUs) employ a shared memory, a software-managed cache for programmers, in each streaming multiprocessor to accelerate data sharing among the threads in a thread block. Although 60% of the shared memory space is underutilized, on average, there are some workloads that demand higher shared memory capacities. Therefore, improving shared memory utilization while satisfying the needs of shared memory intensive workloads is challenging. We make a key observation that the lifetime of each shared memory address is significantly shorter than the execution time of a thread block. In this paper, we first propose Off-Chip Shared Memory (OSM) that allocates shared memory space in the off-chip memory, and accelerates accesses to it via a small on-chip cache. Using an 8 KB cache for shared memory addresses, OSM provides almost the same performance as the baseline GPU that uses 96 KB on-chip shared memory. OSM improves GPU performance in two ways. First, it allocates higher shared memory capacities in the off-chip memory, and improves thread-level parallelism (TLP). Second, it designs a unified cache for shared memory and global address spaces, providing more caching space for global memory address space even for the workloads with high shared memory utilization. Our experimental results show an average 21% and 18% IPC improvement compared to the baseline and the state-of-the-art architectures.

Improving GPU performance in multimedia applications through FPGA based adaptive DMA controller

Purpose Deep learning techniques are unavoidable in a variety of domains such as health care, computer vision, cyber-security and so on. These algorithms demand high data transfers but require bottlenecks in achieving the high speed and low latency synchronization while being implemented in the real hardware architectures. Though direct memory access controller (DMAC) has gained a brighter light of research for achieving bulk data transfers, existing direct memory access (DMA) systems continue to face the challenges of achieving high-speed communication. The purpose of this study is to develop an adaptive-configured DMA architecture for bulk data transfer with high throughput and less time-delayed computation. Design/methodology/approach The proposed methodology consists of a heterogeneous computing system integrated with specialized hardware and software. For the hardware, the authors propose an field programmable gate array (FPGA)-based DMAC, which transfers the data to the graphics processing unit (GPU) using PCI-Express. The workload characterization technique is designed using Python software and is implementable for the advanced risk machine Cortex architecture with a suitable communication interface. This module offloads the input streams of data to the FPGA and initiates the FPGA for the control flow of data to the GPU that can achieve efficient processing. Findings This paper presents an evaluation of a configurable workload-based DMA controller for collecting the data from the input devices and concurrently applying it to the GPU architecture, bypassing the hardware and software extraneous copies and bottlenecks via PCI Express. It also investigates the usage of adaptive DMA memory buffer allocation and workload characterization techniques. The proposed DMA architecture is compared with the other existing DMA architectures in which the performance of the proposed DMAC outperforms traditional DMA by achieving 96% throughput and 50% less latency synchronization. Originality/value The proposed gated recurrent unit has produced 95.6% accuracy in characterization of the workloads into heavy, medium and normal. The proposed model has outperformed the other algorithms and proves its strength for workload characterization.

High-Performance 32-Bit Parallel Hybrid Adder Design Using RNS and Hybrid PTL/CMOS Logic

Adders are one of the essential blocks of Arithmetic Logic Unit (ALU), addressing the memory, table indices and many more such types of applications. The speed of the adder unit more often decides the performance of CPU (Central Processing Unit) and GPU (Graphics Processing Unit) for graphics applications. The high-speed design is a very important performance parameter speed that too with less implementation area and low power consumption. In this paper, the author proposes a novel 32-bit Residue Hybrid Adder (RHA) using the Residue Number System (RNS) and implemented using a Hybrid CMOS/PTL logic style. An RNS has the advantage of representing a large integer using a set of few smaller integers to make computation more efficient and effective. On the other side, parallel prefix adders provide faster execution time as it performs the operation in parallel. With our paper, it is evident that RHA gives better performance in terms of delay, power consumption and area for arithmetic operations. The experimental analysis has been performed using the EDA tool on 45-nm CMOS technology. The power, delay and power delay product (PDP) performance parameters are compared with the existing adders and the results show that, thanks to smaller modules, proposed units have both smaller area and delay by up to 45% and 41%, and, consequently, they allow achieving up to over 46% power saving, respectively.

Quokka: a code for two-moment AMR radiation hydrodynamics on GPUs

ABSTRACT We present quokka, a new subcycling-in-time, block-structured adaptive mesh refinement (AMR) radiation hydrodynamics (RHD) code optimized for graphics processing units (GPUs). quokka solves the equations of HD with the piecewise parabolic method (PPM) in a method-of-lines formulation, and handles radiative transfer via the variable Eddington tensor (VET) radiation moment equations with a local closure. We use the amrex library to handle the AM management. In order to maximize GPU performance, we combine explicit-in-time evolution of the radiation moment equations with the reduced speed-of-light approximation. We show results for a wide range of test problems for HD, radiation, and coupled RHD. On uniform grids in 3D on a single GPU, our code achieves >250 million hydrodynamic updates per second and almost 40 million radiation hydrodynamic updates per second. For RHD problems on uniform grids in 3D, our code scales from 4 to 256 GPUs with an efficiency of 76 per cent. The code is publicly released under an open-source license on GitHub.

Ginkgo : A Modern Linear Operator Algebra Framework for High Performance Computing

In this article, we present Ginkgo , a modern C++ math library for scientific high performance computing. While classical linear algebra libraries act on matrix and vector objects, Ginkgo ’s design principle abstracts all functionality as “linear operators,” motivating the notation of a “linear operator algebra library.” Ginkgo ’s current focus is oriented toward providing sparse linear algebra functionality for high performance graphics processing unit (GPU) architectures, but given the library design, this focus can be easily extended to accommodate other algorithms and hardware architectures. We introduce this sophisticated software architecture that separates core algorithms from architecture-specific backends and provide details on extensibility and sustainability measures. We also demonstrate Ginkgo ’s usability by providing examples on how to use its functionality inside the MFEM and deal.ii finite element ecosystems. Finally, we offer a practical demonstration of Ginkgo ’s high performance on state-of-the-art GPU architectures.

Artificial intelligence: machine learning for chemical sciences.

Research in molecular sciences witnessed the rise and fall of Artificial Intelligence (AI)/ Machine Learning (ML) methods, especially artificial neural networks, few decades ago. However, we see a major resurgence in the use of modern ML methods in scientific research during the last few years. These methods have had phenomenal success in the areas of computer vision, speech recognition, natural language processing (NLP), etc. This has inspired chemists and biologists to apply these algorithms to problems in natural sciences. Availability of high performance Graphics Processing Unit (GPU) accelerators, large datasets, new algorithms, and libraries has enabled this surge. ML algorithms have successfully been applied to various domains in molecular sciences by providing much faster and sometimes more accurate solutions compared to traditional methods like Quantum Mechanical (QM) calculations, Density Functional Theory (DFT) or Molecular Mechanics (MM) based methods, etc. Some of the areas where the potential of ML methods are shown to be effective are in drug design, prediction of high–level quantum mechanical energies, molecular design, molecular dynamics materials, and retrosynthesis of organic compounds, etc. This article intends to conceptually introduce various modern ML methods and their relevance and applications in computational natural sciences.Graphical abstract Synopsis Recent surge in the application of machine learning (ML) methods in fundamental sciences has led to a perspective that these methods may become important tools in chemical science. This perspective provides an overview of the modern ML methods and their successful applications in chemistry during the last few years.

A Neural Network-Inspired Matrix Formulation of Chemical Kinetics for Acceleration on GPUs

High-fidelity simulations of turbulent flames are computationally expensive when using detailed chemical kinetics. For practical fuels and flow configurations, chemical kinetics can account for the vast majority of the computational time due to the highly non-linear nature of multi-step chemistry mechanisms and the inherent stiffness of combustion chemistry. While reducing this cost has been a key focus area in combustion modeling, the recent growth in graphics processing units (GPUs) that offer very fast arithmetic processing, combined with the development of highly optimized libraries for artificial neural networks used in machine learning, provides a unique pathway for acceleration. The goal of this paper is to recast Arrhenius kinetics as a neural network using matrix-based formulations. Unlike ANNs that rely on data, this formulation does not require training and exactly represents the chemistry mechanism. More specifically, connections between the exact matrix equations for kinetics and traditional artificial neural network layers are used to enable the usage of GPU-optimized linear algebra libraries without the need for modeling. Regarding GPU performance, speedup and saturation behaviors are assessed for several chemical mechanisms of varying complexity. The performance analysis is based on trends for absolute compute times and throughput for the various arithmetic operations encountered during the source term computation. The goals are ultimately to provide insights into how the source term calculations scale with the reaction mechanism complexity, which types of reactions benefit the GPU formulations most, and how to exploit the matrix-based formulations to provide optimal speedup for large mechanisms by using sparsity properties. Overall, the GPU performance for the species source term evaluations reveals many informative trends with regards to the effect of cell number on device saturation and speedup. Most importantly, it is shown that the matrix-based method enables highly efficient GPU performance across the board, achieving near-peak performance in saturated regimes.

Efficient GPU-parallelization of batch plants design using metaheuristics with parameter tuning

We address a practice-relevant optimization problem: optimizing multi-product batch plants, with a real-world use case study – optimal design of chemical-engineering systems. Our contribution is a novel approach to parallelizing this optimization problem on GPU (Graphics Processing Units) by combining two metaheuristics – Simulated Annealing (SA) and Ant Colony Optimization (ACO). We improve the implementation performance by tuning particular parameters of the ACO metaheuristic. Our tuning approach improves on the previous methods in two respects: (1) we do not have to rely on additional mechanisms like fuzzy logic or algorithms for online tuning; and (2) we use the high computation performance of GPU to speedup the tuning process. By parallelizing the tuning process on modern GPUs, we allow the user to experiment with large volumes of input data and find the optimal values of the ACO parameters in feasible time. Our experiments on NVIDIA GPU show the efficiency of our approach to parameter tuning for the ACO metaheuristic.

Architecture exploration of recent GPUs to analyze the efficiency of hardware resources

This study analyzes the efficiency of parallel computational applications with the adoption of recent graphics processing units (GPUs). We investigate the impacts of the additional resources of recent architecture on the popular benchmarks compared with previous architecture. Our simulation results demonstrate that Pascal GPU architecture improves the performance by 273% on average compared to old-fashioned Fermi architecture. To evaluate the performance improvement depending on specific hardware resources, we divide the hardware resources into two types: computing and memory resources. Computing resources have bigger impact on performance improvement than memory resources in most of benchmarks. For Hotspot and B+ tree, the architecture adopting only enhanced computing resources can achieve similar performance gains of the architecture adopting both computing and memory resources. We also evaluate the influence of the number of warp schedulers in the SM (Streaming Multiprocessor) to the GPU performance in relationship with barrier waiting time. Based on these analyses, we propose the development direction for the future generation of GPUs.

New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs

On-chip caches of graphics processing units (GPUs) have contributed to improved GPU performance by reducing long memory access latency. However, cache efficiency remains low despite the facts that recent GPUs have considerably mitigated the bottleneck problem of L1 data cache. Although the cache miss rate is a reasonable metric for cache efficiency, it is not necessarily proportional to GPU performance. In this study, we introduce a second key determinant to overcome the problem of predicting the performance gains from L1 data cache based on the assumption that miss rate only is not accurate. The proposed technique estimates the benefits of the cache by measuring the balance between cache efficiency and throughput. The throughput of the cache is predicted based on the warp occupancy information in the warp pool. Then, the warp occupancy is used for a second bypass phase when workloads show an ambiguous miss rate. In our proposed architecture, the L1 data cache is turned off for a long period when the warp occupancy is not high. Our two-level bypassing technique can be applied to recent GPU models and improves the performance by 6% on average compared to the architecture without bypassing. Moreover, it outperforms the conventional bottleneck-based bypassing techniques.