NVIDIA's Architecture Research Articles

BLAS Kütüphanelerinin GPU Mimarilerindeki Nicel Performans Analizi

Basic Linear Algebra Subprograms (BLAS) are a set of linear algebra routines commonly used by machine learning applications and scientific computing. BLAS libraries with optimized implementations of BLAS routines offer high performance by exploiting parallel execution units in target computing systems. With massively large number of cores, graphics processing units (GPUs) exhibit high performance for computationally-heavy workloads. Recent BLAS libraries utilize parallel cores of GPU architectures efficiently by employing inherent data parallelism. In this study, we analyze GPU-targeted functions from two BLAS libraries, cuBLAS and MAGMA, and evaluate their performance on a single-GPU NVIDIA architecture by considering architectural features and limitations. We collect architectural performance metrics and explore resource utilization characteristics. Our work aims to help researchers and programmers to understand the performance behavior and GPU resource utilization of the BLAS routines implemented by the libraries.

Parallelization of numerical solutions of shallow water equations by the finite volume method for implementation on multiprocessor systems and graphics processors

An overview of approaches to parallelization of grid-based numerical methods for solving shallow water equations for multiprocessor systems and graphics processors is presented. A multithreaded approach for shared-memory computing systems implemented on the basis of the OpenMP programming interface and a geometric decomposition approach with message-passing using the MPI library for distributed-memory computers are described. Multithreading for programming GPUs based on the OpenACC software interface is considered. For the COASTOX-UN system of two-dimensional modeling of hydrodynamics, sediment and radionuclide transport in river systems and coastal areas of the seas, the parallelization of its hydrodynamic model COASTOX-HD was carried out. In the developed numerical model, the shallow water equations are solved by finite-volume numerical methods on unstructured computational grids with triangular cells of variable size. The parallelization is implemented using a hybrid MPI+OpenACC approach targeting multiprocessor systems and GPUs. For multiprocessor computers, geometric decomposition and MPI-based messaging are used, and for GPUs, multithreading is implemented using OpenACC directives. The performance of the developed parallel hydrodynamic model was evaluated during the calculation of typical problems of hydrodynamics of shallow water bodies, river flood, and tsunami wave run-up on the coast on a Dell Precision Workstation 7920 multi-core workstation with two 20-core Intel Xeon Gold 6230 processors and NVIDIA Quadro RTX 5000 and NVIDIA GeForce RTX 3080 graphics cards. It is shown that the developed model has significantly accelerated the simulation on the considered multiprocessor system and the considered GPUs. The acceleration on GPUs depends on the size of the computational grid, increasing to saturation with an increase in the number of grid cells. It is established that for the developed parallel model, whose numerical schemes are related to algorithms with low computational intensity, the memory bandwidth of the NVIDIA architecture GPUs is a more important limiting factor of acceleration than their performance.

GPU acceleration of local and semilocal density functional calculations in the SPARC electronic structure code.

We present a Graphics Processing Unit (GPU)-accelerated version of the real-space SPARC electronic structure code for performing Kohn-Sham density functional theory calculations within the local density and generalized gradient approximations. In particular, we develop a modular math-kernel based implementation for NVIDIA architectures wherein the computationally expensive operations are carried out on the GPUs, with the remainder of the workload retained on the central processing units (CPUs). Using representative bulk and slab examples, we show that relative to CPU-only execution, GPUs enable speedups of up to 6× and 60× in node and core hours, respectively, bringing time to solution down to less than 30s for a metallic system with over 14 000 electrons and enabling significant reductions in computational resources required for a given wall time.

Fine‐Grained Memory Profiling of GPGPU Kernels

AbstractMemory performance is a crucial bottleneck in many GPGPU applications, making optimizations for hardware and software mandatory. While hardware vendors already use highly efficient caching architectures, software engineers usually have to organize their data accordingly in order to efficiently make use of these, requiring deep knowledge of the actual hardware. In this paper we present a novel technique for fine‐grained memory profiling that simulates the whole pipeline of memory flow and finally accumulates profiling values in a way that the user retains information about the potential region in the GPU program by showing these values separately for each allocation. Our memory simulator turns out to outperform state‐of‐the‐art memory models of NVIDIA architectures by a magnitude of 2.4 for the L1 cache and 1.3 for the L2 cache, in terms of accuracy. Additionally, we find our technique of fine grained memory profiling a useful tool for memory optimizations, which we successfully show in case of ray tracing and machine learning applications.

Billion degree of freedom granular dynamics simulation on commodity hardware via heterogeneous data-type representation

We discuss modeling, algorithmic, and software aspects that allow a simulation tool called Chrono::Granular to run billion-degree-of-freedom dynamics problems on commodity hardware, i.e., a workstation with one GPU. The ability to scale the solution to large problem sizes is traced back to an adimensionalization process combined with the use of mixed-precision data types that reduce memory pressure and improve arithmetic intensity, judicious use of the memory ecosystem on GPU cards as exposed by CUDA on Nvidia architectures, and a software implementation that prioritizes execution speed over modeling generality. The simulation approach is demonstrated for 3D scenarios with up to 710 million bodies for the frictionless case (of relevance in emulsions), and up to 210 million bodies for scenarios with friction (of relevance in terradynamics, additive manufacturing, soft-matter physics). The frictional contact model used draws on the Discrete Element Method (DEM). A performance benchmark shows linear scaling with problem size up to GPU memory capacity. The implementation has an application programming interface that enables it to interact in a cosimulation framework with third-party dynamics engines. This interaction is anchored by a force–displacement data exchange protocol that brings in external bodies as geometries defined by triangle meshes. We demonstrate the cosimulation mechanism by interfacing to an open source, multiphysics simulation engine called Chrono. Therein, triangular meshes define moving boundary conditions for Chrono::Granular, which in turn provides forces and torques acting on the triangular meshes. Several tests are considered for validation and scaling analysis purposes. The limiting aspects of the current implementation are its exclusive support of monodisperse granular systems, and its lack of handling geometries beyond spheres. These limitations are addressed by ongoing work.

High-throughput fuzzy clustering on heterogeneous architectures

The Internet of Things (IoT) is pushing the next economic revolution in which the main players are data and immediacy. IoT is increasingly producing large amounts of data that are now classified as “dark data” because most are created but never analyzed. The efficient analysis of this data deluge is becoming mandatory in order to transform it into meaningful information. Among the techniques available for this purpose, clustering techniques, which classify different patterns into groups, have proven to be very useful for obtaining knowledge from the data. However, clustering algorithms are computationally hard, especially when it comes to large data sets and, therefore, they require the most powerful computing platforms on the market. In this paper, we investigate coarse and fine grain parallelization strategies in Intel and Nvidia architectures of fuzzy minimals (FM) algorithm; a fuzzy clustering technique that has shown very good results in the literature. We provide an in-depth performance analysis of the FM’s main bottlenecks, reporting a speed-up factor of up to 40× compared to the sequential counterpart version.

Comparison of analytical and ML-based models for predicting CPU–GPU data transfer time

The overhead of data transfer to the GPU poses a bottleneck for the performance of CUDA programs. The accurate prediction of data transfer time is quite effective in improving the performance of GPU analytical modeling, the prediction accuracy of kernel performance, and the composition of the CPU with the GPU for solving computational problems. For estimating the data transfer time between the CPU and the GPU, the current study employs three machine learning-based models and a new analytical model called $$\lambda $$-Model. These models run on four GPUs from different NVIDIA architectures and their performance is compared. The practical results show that the $$\lambda $$-Model is able to anticipate the transmission of large-sized data with a maximum error of 1.643%, which offers better performance than that of machine learning methods. As for the transmission of small-sized data, machine learning-based methods provide better performance and a predicted data transfer time with a maximum error of 4.52%. Consequently, the current study recommends a hybrid model, that is, the $$\lambda $$-Model for large-sized data and machine learning tools for small-sized data.

Exploiting Bank Conflict-based Side-channel Timing Leakage of GPUs

To prevent information leakage during program execution, modern software cryptographic implementations target constant-time function, where the number of instructions executed remains the same when program inputs change. However, the underlying microarchitecture behaves differently when processing different data inputs, impacting the execution time of the same instructions. These differences in execution time can covertly leak confidential information through a timing channel. Given the recent reports of covert channels present on commercial microprocessors, a number of microarchitectural features on CPUs have been re-examined from a timing leakage perspective. Unfortunately, a similar microarchitectural evaluation of the potential attack surfaces on GPUs has not been adequately performed. Several prior work has considered a timing channel based on the behavior of a GPU’s coalescing unit. In this article, we identify a second finer-grained microarchitectural timing channel, related to the banking structure of the GPU’s Shared Memory. By considering the timing channel caused by Shared Memory bank conflicts, we have developed a differential timing attack that can compromise table-based cryptographic algorithms. We implement our timing attack on an Nvidia Kepler K40 GPU and successfully recover the complete 128-bit encryption key of an Advanced Encryption Standard (AES) GPU implementation using 900,000 timing samples. We also evaluate the scalability of our attack method by attacking an implementation of the AES encryption algorithm that fully occupies the compute resources of the GPU. We extend our timing analysis onto other Nvidia architectures: Maxwell, Pascal, Volta, and Turing GPUs. We also discuss countermeasures and experiment with a novel multi-key implementation, evaluating its resistance to our side-channel timing attack and its associated performance overhead.

ИССЛЕДОВАНИЕ МОДЕЛЕЙ РЕАЛИЗАЦИИ ВОЛНОВОГО АЛГОРИТМА ДВИЖЕНИЯ РОБОТА ДЛЯ АРХИТЕКТУРЫ NVIDIA В ТЕХНОЛОГИИ CUDA

ИССЛЕДОВАНИЕ МОДЕЛЕЙ РЕАЛИЗАЦИИ ВОЛНОВОГО АЛГОРИТМА ДВИЖЕНИЯ РОБОТА ДЛЯ АРХИТЕКТУРЫ NVIDIA В ТЕХНОЛОГИИ CUDA

DawnCC

Directive-based programming models, such as OpenACC and OpenMP, allow developers to convert a sequential program into a parallel one with minimum human intervention. However, inserting pragmas into production code is a difficult and error-prone task, often requiring familiarity with the target program. This difficulty restricts the ability of developers to annotate code that they have not written themselves. This article provides a suite of compiler-related methods to mitigate this problem. Such techniques rely on symbolic range analysis, a well-known static technique, to achieve two purposes: populate source code with data transfer primitives and to disambiguate pointers that could hinder automatic parallelization due to aliasing. We have materialized our ideas into a tool, DawnCC, which can be used stand-alone or through an online interface. To demonstrate its effectiveness, we show how DawnCC can annotate the programs available in PolyBench without any intervention from users. Such annotations lead to speedups of over 100× in an Nvidia architecture and over 50× in an ARM architecture.

Efficient implementation of morphological index for building/shadow extraction from remotely sensed images

Morphological building index (MBI) and morphological shadow index (MSI) are recently developed techniques that aim at automatically detect buildings/shadows using high-resolution remotely sensed imagery. The traditional mathematical morphology operations are usually time-consuming as they are based on the consideration of a wide range of image-object properties, such as brightness, contrast, shapes, sizes, and in the application of series of repeated transformations (e.g., classical opening and closing operators). In the case of MBI and MSI, the computational complexity is also increased due to the use of multiscale and multidirectional morphological operators. In this paper, we provide a computationally efficient implementation of MBI and MSI algorithms which is specifically developed for commodity graphic processing units using NVIDIA CUDA. We perform the evaluation of the parallel version of the algorithms using two different NVIDIA architectures and three widely used hyperspectral data sets. Experimental results show that the computational burden introduced when considering multidirectional morphological operators can be almost completely removed by the developed implementations.

The Advanced Synthetic Aperture Sonar Imaging eNgine (ASASIN), a time-domain backprojection beamformer using graphics processing units

We present a synthetic aperture sonar beamforming engine called ASASIN. ASASIN is a graphics processing unit based time-domain backprojection beamformer for Windows and Linux utilizing the NVIDIA architecture. It supports arbitrary array geometries and has a modular data input system. ASASIN uses one or more graphics processing units to beamform synthetic aperture sonar data faster than real-time. Its interface is suitable for operators and research scientists. In this talk we discuss ASASINs algorithms and computational performance along with the choices confronted (both algorithmic and computational) in its design. Finally, we discuss its computational performance across a variety of NVIDIA graphics processing units including their embedded models.

Implementation of Sorting Algorithms with CUDA: An Empirical Study

Sorting algorithms have been studied for more than 3 decades now. The aim of this paper is to implement some of the sorting algorithms using the CUDA language in a GPU environment provided by the Nvidia graphics cards. This empirical study is done for comparing the performance of the sorting algorithms in a run-time environment provided by the GPUs and the CUDA programming language. This study considers the implementation of bubble sort, insertion sort, quicksort, selection sort and shell sort algorithms. It is shown that there is a significant amount of speed-up in using CUDA and the Nvidia architecture instead of a sequential code running on standard architectures.

Exposing errors related to weak memory in GPU applications

We present the systematic design of a testing environment that uses stressing and fuzzing to reveal errors in GPU applications that arise due to weak memory effects. We evaluate our approach on seven GPUs spanning three Nvidia architectures, across ten CUDA applications that use fine-grained concurrency. Our results show that applications that rarely or never exhibit errors related to weak memory when executed natively can readily exhibit these errors when executed in our testing environment. Our testing environment also provides a means to help identify the root causes of such errors, and automatically suggests how to insert fences that harden an application against weak memory bugs. To understand the cost of GPU fences, we benchmark applications with fences provided by the hardening strategy as well as a more conservative, sound fencing strategy.

Evaluation of the 3-D finite difference implementation of the acoustic diffusion equation model on massively parallel architectures

The diffusion equation model is a popular tool in room acoustics modeling. The 3-D Finite Difference (3D-FD) implementation predicts the energy decay function and the sound pressure level in closed environments. This simulation is computationally expensive, as it depends on the resolution used to model the room. With such high computational requirements, a high-level programming language (e.g., Matlab) cannot deal with real life scenario simulations. Thus, it becomes mandatory to use our computational resources more efficiently. Manycore architectures, such as NVIDIA GPUs or Intel Xeon Phi offer new opportunities to enhance scientific computations, increasing the performance per watt, but shifting to a different programming model. This paper shows the roadmap to use massively parallel architectures in a 3D-FD simulation. We evaluate the latest generation of NVIDIA and Intel architectures. Our experimental results reveal that NVIDIA architectures outperform by a wide margin the Intel Xeon Phi co-processor while dissipating approximately 50W less (25%) for large-scale input problems.

Comprehensive Evaluation of a New GPU-based Approach to the Shortest Path Problem

The single-source shortest path (SSSP) problem arises in many different fields. In this paper, we present a GPU SSSP algorithm implementation. Our work significantly speeds up the computation of the SSSP, not only with respect to a CPU-based version, but also to other state-of-the-art GPU implementations based on Dijkstra. Both GPU implementations have been evaluated using the latest NVIDIA architectures. The graphs chosen as input sets vary in nature, size, and fan-out degree, in order to evaluate the behavior of the algorithms for different data classes. Additionally, we have enhanced our GPU algorithm implementation using two optimization techniques: The use of a proper choice of threadblock size; and the modification of the GPU L1 cache memory state of NVIDIA devices. These optimizations lead to performance improvements of up to 23 % with respect to the non-optimized versions. In addition, we have made a platform comparison of several NVIDIA boards in order to distinguish which one is better for each class of graphs, depending on their features. Finally, we compare our results with an optimized sequential implementation of Dijkstra's algorithm included in the reference Boost library, obtaining an improvement ratio of up to 19$$\times $$× for some graph families, using less memory space.

Execution time optimisation using delayed multidimensional retiming

Multidimensional retiming MR is a software pipelining approach that ensures increasing the instruction-level parallelism across all the nested loops. All the MR techniques aim at achieving a full parallelism in order to schedule applications with a minimal cycle period. However, the growth of code sizes in terms of parallelism level engenders the rise in cycle period numbers. Thus, fully parallel multidimensional applications frequently face limiting factors when implemented on real-time systems. This paper presents a novel technique, called delayed MR, which schedules nested loops with a minimal cycle period, without achieving full parallelism. It is formulated into two efficient steps whose first one sweeps the nested loops with the target of selecting and ordering paths, whereas the second one applies an optimal MR to the selected paths. Our technique is verified by implementing several nested loops in NVIDIA architectures. The experimental results show that our technique achieves average improvements on execution time of 32.8% compared to the incremental technique and 19.35% compared to the chained one.

SEARCH-BASED AUTOMATIC CODE GENERATION FOR MULTIPRECISION MODULAR EXPONENTIATION ON MULTIPLE GENERATIONS OF GPU

Multiprecision modular exponentiation has a variety of uses, including cryptography, prime testing and computational number theory. It is also a very costly operation to compute. GPU parallelism can be used to accelerate these computations, but to use the GPU efficiently, a problem must involve many simultaneous exponentiation operations. Handling a large number of TLS/SSL encrypted sessions in a data center is an important problem that fits this profile. We are developing a framework that enables generation of highly efficient implementations of exponentiation operations for different NVIDIA GPU architectures and problem instances. One of the challenges in generating such code is that NVIDIA's PTX is not a true assembly language, but is instead a virtual instruction set that is compiled and optimized in different ways for different generations of GPU hardware. Thus, the same PTX code runs with different levels of efficiency on different machines. And as the precision of the computations changes, each architecture has its own break-even points where a different algorithm or parallelization strategy must be employed. To make the code efficient for a given problem instance and architecture thus requires searching a multidimensional space of algorithms and configurations, by generating PTX code for each combination, executing it, validating the numerical result, and evaluating its performance. Our framework automates much of this process, and produces exponentiation code that is up to six times faster than the best known hand-coded implementations for the NVIDIA GTX 580. Our goal for the framework is to enable users to relatively quickly find the best configuration for each new GPU architecture. However, in migrating to the GTX 680, which has three times as many cores as the GTX 580, we found that the best performance our system could achieve was significantly less than for the GTX 580. The decrease was traced to a radical shift in the NVIDIA architecture that greatly reduces the storage resources for each core. Further analysis and feasibility simulations indicate that it should be possible, through changes in our code generators to adapt for different storage models, to take greater advantage of the parallelism on the GTX 680. That will add a new dimension to our search space, but will also give our framework greater flexibility for dealing with future architectures.