Graphics Processing Units Applications Research Articles

Accelerating Conjugate Heat Transfer Simulations in Squared Heated Cavities through Graphics Processing Unit (GPU) Computing

This research develops an innovative framework for accelerating Conjugate Heat Transfer (CHT) simulations within squared heated cavities through the application of Graphics Processing Units (GPUs). Although leveraging GPUs for computational speed improvements is well recognized, this study distinguishes itself by formulating a tailored optimization strategy utilizing the CUDA-C programming language. This approach is specifically designed to tackle the inherent challenges of modeling squared cavity configurations in thermal simulations. Comparative performance evaluations reveal that our GPU-accelerated framework reduces computation times by up to 99.7% relative to traditional mono-core CPU processing. More importantly, it demonstrates an increase in accuracy in heat transfer predictions compared to existing CPU-based models. These results highlight not only the technical feasibility but also the substantial enhancements in simulation efficiency and accuracy, which are crucial for critical engineering applications such as aerospace component design, electronic device cooling, and energy system optimization. By advancing GPU computational techniques, this work contributes significantly to the field of thermal management, offering a potential for broader application and paving the way for more efficient, sustainable engineering solutions.

Predictable GPU Wavefront Splitting for Safety-Critical Systems

We present a predictable wavefront splitting (PWS) technique for graphics processing units (GPUs). PWS improves the performance of GPU applications by reducing the impact of branch divergence while ensuring that worst-case execution time (WCET) estimates can be computed. This makes PWS an appropriate technique to use in safety-critical applications, such as autonomous driving systems, avionics, and space, that require strict temporal guarantees. In developing PWS on an AMD-based GPU, we propose microarchitectural enhancements to the GPU, and a compiler pass that eliminates branch serializations to reduce the WCET of a wavefront. Our analysis of PWS exhibits a performance improvement of 11% over existing architectures with a lower WCET than prior works in wavefront splitting.

Program Analysis and Machine Learning–based Approach to Predict Power Consumption of CUDA Kernel

The General Purpose Graphics Processing Unit has secured a prominent position in the High-Performance Computing world due to its performance gain and programmability. Understanding the relationship between Graphics Processing Unit (GPU) power consumption and program features can aid developers in building energy-efficient sustainable applications. In this work, we propose a static analysis-based power model built using machine learning techniques. We have investigated six machine learning models across three NVIDIA GPU architectures: Kepler, Maxwell, and Volta with Random Forest, Extra Trees, Gradient Boosting, CatBoost, and XGBoost reporting favorable results. We observed that the XGBoost technique-based prediction model is the most efficient technique with an R 2 value of 0.9646 on Volta Architecture. The dataset used for these techniques includes kernels from different benchmarks suits, sizes, nature (e.g., compute-bound, memory-bound), and complexity (e.g., control divergence, memory access patterns). Experimental results suggest that the proposed solution can help developers precisely predict GPU applications power consumption using program analysis across GPU architectures. Developers can use this approach to refactor their code to build energy-efficient GPU applications.

RTGPU: Real-Time GPU Scheduling of Hard Deadline Parallel Tasks With Fine-Grain Utilization

Many emerging cyber-physical systems, such as autonomous vehicles and robots, rely heavily on artificial intelligence and machine learning algorithms to perform important system operations. Since these highly parallel applications are computationally intensive, they need to be accelerated by graphics processing units (GPUs) to meet stringent timing constraints. However, despite the wide adoption of GPUs, efficiently scheduling multiple GPU applications while providing rigorous real-time guarantees remains a challenge. Each GPU application has multiple CPU execution and memory copy segments, with GPU kernels running on different hardware resources. Because of the complicated interactions between heterogeneous segments of parallel tasks, high schedulability is hard to achieve with conventional approaches. This paper proposes RTGPU, which combines fine-grain GPU partitioning on the system side with a novel scheduling algorithm on the theory side. Through system and theory co-design, RTGPU achieves superior system throughput and real-time schedulability. In this paper, we start by building a model for the CPU and memory copy segments. Leveraging persistent threads, we then implement fine-grained GPU partitioning with improved performance through interleaved execution. To reap the benefits of fine-grained GPU partitioning and schedule multiple parallel GPU applications, we propose a novel real-time scheduling algorithm based on federated scheduling and grid search with uniprocessor fixed-priority scheduling. Our approach provides real-time guarantees to meet hard deadlines, and achieves over 11% improvement in system throughput and up to 57% schedulability improvement compared with previous work. We validate and evaluate RTGPU on NVIDIA GTX1080Ti GPU systems. Our system side techniques can be applied on mainstream NVIDIA GPUs, and the proposed scheduling theory can be used in general heterogeneous computing platforms which have a similar task execution pattern.

Computing large 2D convolutions on GPU efficiently with the im2tensor algorithm

Attaining the best possible throughput when computing convolutions is a challenge for signal and image processing systems, be they HPC (High-Performance Computing) machines or embedded real-time targets. This importance is highlighted by the numerous methods and implementations available, often optimized for particular settings: small batched kernels or very large kernels, for example. In the meantime, GPUs (Graphics Processing Units) have become a first-class architecture for real-time and embedded processing. The power offered by those chips stems from their parallel nature, and this advantage has been exploited for convolutions in several libraries. Even more recently, the introduction of tensor cores on NVIDIA GPUs has opened up new limits in terms of attainable FLOPS (Floating-Point Operations per Second). For reaching that performance, GPU applications must use GEMMs (GEneral Matrix Multiplications), that tensor cores accelerate. We then developed an efficient GEMM-based 2D convolution algorithm in a general setting. On relatively large kernels (30–50-pixel wide), im2tensor is, to the best of our knowledge, the fastest method for computing 2D convolutions. We provide detailed performance analysis for different scenarios: small (1024\(\times\)1024) and large (4096\(\times\)4096) images, with convolutions kernels of sizes ranging 1 to 60-pixel wide, on two GPU cards: Jetson AGX Xavier (embedded) and Titan V (server-class). Moreover, the accuracy of im2tensor surpasses non-GEMM based methods, thanks to the larger-precision registers used by tensor cores for intermediate representations.

High-Performance Image Acquisition and Processing for Stereoscopic Diagnostic Systems with the Application of Graphical Processing Units.

In recent years, cinematography and other digital content creators have been eagerly turning to Three-Dimensional (3D) imaging technology. The creators of movies, games, and augmented reality applications are aware of this technology’s advantages, possibilities, and new means of expression. The development of electronic and IT technologies enables the achievement of a better and better quality of the recorded 3D image and many possibilities for its correction and modification in post-production. However, preparing a correct 3D image that does not cause perception problems for the viewer is still a complex and demanding task. Therefore, planning and then ensuring the correct parameters and quality of the recorded 3D video is essential. Despite better post-production techniques, fixing errors in a captured image can be difficult, time consuming, and sometimes impossible. The detection of errors typical for stereo vision related to the depth of the image (e.g., depth budget violation, stereoscopic window violation) during the recording allows for their correction already on the film set, e.g., by different scene layouts and/or different camera configurations. The paper presents a prototype of an independent, non-invasive diagnostic system that supports the film crew in the process of calibrating stereoscopic cameras, as well as analysing the 3D depth while working on a film set. The system acquires full HD video streams from professional cameras using Serial Digital Interface (SDI), synchronises them, and estimates and analyses the disparity map. Objective depth analysis using computer tools while recording scenes allows stereographers to immediately spot errors in the 3D image, primarily related to the violation of the viewing comfort zone. The paper also describes an efficient method of analysing a 3D video using Graphics Processing Unit (GPU). The main steps of the proposed solution are uncalibrated rectification and disparity map estimation. The algorithms selected and implemented for the needs of this system do not require knowledge of intrinsic and extrinsic camera parameters. Thus, they can be used in non-cooperative environments, such as a film set, where the camera configuration often changes. Both of them are implemented with the use of a GPU to improve the data processing efficiency. The paper presents the evaluation results of the algorithms’ accuracy, as well as the comparison of the performance of two implementations—with and without the GPU acceleration. The application of the described GPU-based method makes the system efficient and easy to use. The system can process a video stream with full HD resolution at a speed of several frames per second.

Reducing GPU Energy Consumption by Packing Narrow-Width Operands

In this paper, we study the use of an Operand-Width-Aware Register (OWAR) packing mechanism for graphics processing unit (GPU) energy saving. In order to efficiently use the GPU register file (RF), OWAR employs a power gating method to shut down unused register sub-arrays in order to reduce dynamic and leakage energy consumption of RF. As the number of register accesses was reduced due to the packing of the narrow width operands, the dynamic energy dissipation was further decreased. Finally, with the help of RF usage optimized by register packing, OWAR allowed GPUs to support more thread-level parallelism (TLP) through assigning additional thread blocks on streaming multiprocessors (SMs) for general-purpose GPU (GPGPU) applications that suffered from the deficiency of register resources. The extra TLP opens opportunities for hiding more memory latencies and thus reducing the overall execution time, which can lower the overall energy consumption. We evaluated OWAR using a set of representative GPU benchmarks. The experimental results showed that compared to the baseline without optimization, OWAR can reduce the GPGPU’s total energy up to 29.6% and 9.5% on average. In addition, OWAR achieved performance improvement up to 1.97X and 1.18X on average.

Interference-aware execution framework with Co-scheML on GPU clusters

Recently, improving the overall resource utilization through efficient scheduling of applications on graphic processing unit (GPU) clusters has been a concern. Traditional cluster-orchestration platforms providing GPUs exclusively for applications constrain high resource utilization. Co-execution of GPU applications is suggested to utilize limited resources. However, the co-execution of GPU applications without considering their diverse characteristics can lead to their unpredictable performances owing to interference resulting from contention and unbalanced usage of resources among applications. This paper proposes an interference-aware execution framework with Co-scheML for various GPU applications such as high performance computing (HPC), deep learning (DL) training, and DL inference. Various resource-usage characteristics of GPU applications are analyzed and profiled to identify various degrees of their application interference. As interference prediction is challenging owing to the complexity of GPU systems, an interference model is generated by applying defined GPU metrics to machine learning (ML) models. A Co-scheML scheduler deploys applications to minimize the interference using the predicted interference from the constructed model. Experimental results of our framework demonstrated that the resource utilization improved by 24%, the average job completion time (JCT) improved by 23%, and the makespan shortened by 22% on average, compared to baseline schedulers.

IterML: Iterative Machine Learning for Intelligent Parameter Pruning and Tuning in Graphics Processing Units

With the rise of graphics processing units (GPUs), the parallel computing community needs better tools to productively extract performance from the GPU. While modern compilers provide flags to activate different optimizations to improve performance, the effectiveness of such automated optimization has been limited at best. As a consequence, extracting the best performance from an algorithm on a GPU requires significant expertise and manual effort to exploit both spatial and temporal sharing of computing resources. In particular, maximizing the performance of an algorithm on a GPU requires extensive hyperparameter (e.g., thread-block size) selection and tuning. Given the myriad of hyperparameter dimensions to optimize across, the search space of optimizations is extremely large, making it infeasible to exhaustively evaluate. This paper proposes an approach that uses statistical analysis with iterative machine learning (IterML) to prune and tune hyperparameters to achieve better performance. During each iteration, we leverage machine-learning models to guide the pruning and tuning for subsequent iterations. We evaluate our IterML approach on the GPU thread-block size across many benchmarks running on an NVIDIA P100 or V100 GPU. Our experimental results show that our automated IterML approach reduces search effort by 40% to 80% when compared to traditional (non-iterative) ML and that the performance of our (unmodified) GPU applications can improve significantly — between 67% and 95% — simply by changing the thread-block size.

Bypass-Enabled Thread Compaction for Divergent Control Flow in Graphics Processing Units

Graphics processing units (GPUs) employ the single instruction multiple data (SIMD) hardware to run threads in parallel and allow each thread to maintain an arbitrary control flow. Threads running concurrently within a warp may jump to different paths after conditional branches. Such divergent control flow makes some lanes idle and hence reduces the SIMD utilization of GPUs. To alleviate the waste of SIMD lanes, threads from multiple warps can be collected together to improve the SIMD lane utilization by compacting threads into idle lanes. However, this mechanism induces extra barrier synchronizations since warps have to be stalled to wait for other warps for compactions, resulting in that no warps are scheduled in some cases. In this paper, we propose an approach to reduce the overhead of barrier synchronizations induced by compactions. In our approach, a compaction is bypassed by warps whose threads all jump to the same path after branches. Moreover, warps waiting for a compaction can also bypass this compaction when no warps are ready for issuing. In addition, a compaction is canceled if idle lanes can not be reduced via this compaction. The experimental results demonstrate that our approach provides an average improvement of 21% over the baseline GPU for applications with massive divergent branches, while recovering the performance loss induced by compactions by 13% on average for applications with many non-divergent control flows.

Application of Graphics Processing Units for Self-Consistent Modelling of Shallow Water Dynamics and Sediment Transport

In this paper, we describe a numerical algorithm for the self-consistent simulations of surface water and sediment dynamics. The method is based on the original Lagrangian-Eulerian CSPH-TVD approach for solving the Saint-Venant and Exner equations, taking into account the physical factors essential for the understanding of the shallow water and surface soil layer motions, including complex terrain structure and its evolution due to sediment transport. Additional Exner equation for sediment transport has been used for the numerical CSPH-TVD scheme stability criteria definition. By using OpenMP-CUDA and GPUDirect technologies for hybrid computing systems (supercomputers) with several graphic coprocessors (GPUs) interacting with each other via the PCI-E / NVLINK interface we also develop a parallel numerical algorithm for the CSPH-TVD method. The developed parallel version of the algorithm demonstrates high efficiency for various configurations of Nvidia Tesla CPU + GPU computing systems. In particular, maximal speed up is 1800 for a system with four C2070 GPUs compare to the serial version for the CPU. The calculation time on the GPU V100~(Volta architecture) is reduced by 95 times compared to the GPU C2070~(Fermi architecture).

Characterizing and Exploiting Soft Error Vulnerability Phase Behavior in GPU Applications

System reliability has become a first-class design constraint. As the use of Graphics Processing Units (GPU) continues to increase in compute applications, including High Performance Computing (HPC) and safety-critical applications, so do the number of transient faults in GPUs. However, our understanding of the potential impact of transient fault propagation in GPU applications remains limited. This study shows that the resilience characteristics of GPU programs change significantly during program execution and these characteristics show repetitive, time-varying behavior. Interestingly, these repetitive, time-varying, resilience characteristics of GPU programs do not align or correlate well with the performance phases of GPU programs. Furthermore, this work discovers and validates that temporal changes in the vulnerability behavior during a kernel execution tends to coincide with changes in basic block execution paths. Finally, we demonstrate how these observations can be exploited to accelerate the fault injection campaigns for reliability assessment of GPU programs by an order of magnitude and open opportunities for designing other effective resilience mitigation strategies.

Efficient Performance Estimation and Work-Group Size Pruning for OpenCL Kernels on GPUs

Graphic Processing Units (GPUs) play a vital role in state-of-the-art high-performance scientific computing realm and research work towards its performance analysis is crucial but nontrivial. Extant GPU performance models are far from practical use, while fine-grained GPU simulation requires a considerably large time cost. Moreover, massive amounts of designs with various program inputs and parameter settings pose a challenge for efficient performance estimation and tuning of parallel GPU applications. To this end, this article presents a hybrid framework for the efficient performance estimation and work-group size pruning of OpenCL workloads on GPUs. The framework contains a static module used to extract the kernel execution trace from the high-level source code and a dynamical module used to mimic the kernel execution flow to estimate the runtime performance. For the design space pruning, an extra analysis is performed to filter out the redundant work-group sizes with duplicated execution traces and inferior pipelines. The proposed framework does not require any program runs to estimate the performance and find the optimal or near-optimal designs. Experiments on four Commercial Off-The-Shelf (COTS) Nvidia GPUs show that the framework can predict the runtime performance with an average error of 17.04 percent and reduce the program design space by an average of 78.47 percent.

Don't forget about synchronization! Guidelines for using locks on graphics processing units

SummaryHeterogeneous devices are becoming necessary components of high performance computing infrastructures, and the graphics processing unit (GPU) plays an important role in this landscape. Given a problem, the established approach for exploiting the GPU is to design solutions that are parallel, without data dependencies. These solutions are then offloaded to the GPU's massively parallel capability. This design principle often leads to developing applications that cannot maximize GPU hardware utilization. The goal of this article is to challenge this common belief by empirically showing that allowing even simple forms of synchronization enables programmers to design solutions that admit conflicts and achieve better performance. Our experience shows that lock‐based solutions to the k‐means clustering problem, implemented using two well‐known locking strategies, outperform the well‐engineered and parallel KMCUDA on both synthetic and real datasets; with an average 8× faster runtimes across all locking algorithms on a synthetic dataset and 1.7× faster on a real world dataset across all locking algorithms (and max speedups of 71.3× and 2.75×, respectively). We validate these results using a more sophisticated clustering algorithm, namely fuzzy c‐means and summarize our findings by identifying three guidelines to help make concurrency effective when programming GPU applications.

Practical Resilience Analysis of GPGPU Applications in the Presence of Single- and Multi-Bit Faults

Graphics Processing Units (GPUs) have rapidly evolved to enable energy-efficient data-parallel computing for a broad range of scientific areas. While GPUs achieve exascale performance at a stringent power budget, they are also susceptible to soft errors, often caused by high-energy particle strikes, that can significantly affect the application output quality. Understanding the resilience of general purpose GPU (GPGPU) applications is especially challenging because unlike CPU applications, which are mostly single-threaded, GPGPU applications can contain hundreds to thousands of threads, resulting in a tremendously large fault site space in the order of billions, even for some simple applications and even when considering the occurrence of just a single-bit fault. We present a systematic way to progressively prune the fault site space aiming to dramatically reduce the number of fault injections such that assessment for GPGPU application error resilience becomes practical. The key insight behind our proposed methodology stems from the fact that while GPGPU applications spawn a lot of threads, many of them execute the same set of instructions. Therefore, several fault sites are redundant and can be pruned by careful analysis. We identify important features across a set of 10 applications (16 kernels) from Rodinia and Polybench suites and conclude that threads can be primarily classified based on the number of the dynamic instructions they execute. We therefore achieve significant fault site reduction by analyzing only a small subset of threads that are representative of the dynamic instruction behavior (and therefore error resilience behavior) of the GPGPU applications. Further pruning is achieved by identifying the dynamic instruction commonalities (and differences) across code blocks within this representative set of threads, a subset of loop iterations within the representative threads, and a subset of destination register bit positions. The above steps result in a tremendous reduction of fault sites by up to seven orders of magnitude. Yet, this reduced fault site space accurately captures the error resilience profile of GPGPU applications. We show the effectiveness of the proposed progressive pruning technique for a single-bit model and illustrate its application to even more challenging cases with three distinct multi-bit fault models.

Toward OS-Level and Device-Level Cooperative Scheduling for Multitasking GPUs

As one of the most popular accelerators, the graphics processing unit (GPU) has been extensively adopted throughout the world. With the burst of new applications and the growing scale of data, co-running applications on limited GPU resources has become increasingly important due to its dramatic improvement in overall system efficiency. Quality of service (QoS) support among concurrent general-purpose GPU (GPGPU) applications is currently one of the most trending research topics. Prior efforts have been focused on providing QoS support either with OS-level or device-level scheduling methods. Each of these scheduling methods possesses pros and cons and may be unable to independently cover all the scheduling cases. In this paper, we propose a cooperative QoS scheduling scheme (C-QoS) that consists of operating-system-level (OS-level) scheduling and device-level scheduling. Our proposed scheme can control the progress of a kernel and provide thorough QoS support for concurrent applications in multitasking GPUs. Due to the accurate resource management of the copy engine and execution engine, C-QoS achieves QoS goals 23.33% more often than state-of-the-art QoS support mechanisms. The results demonstrate that cooperative methods achieve 17.27% higher system utilization than uncooperative methods.

Kernel-Based Resource Allocation for Improving GPU Throughput While Minimizing the Activity Divergence of SMs

Graphics Processing Units (GPUs) have been established as a major part of modern computing systems. As technology scales down, GPUs integrate more computing elements that accelerate massively parallel applications. Due to the increase of GPU cores, sophisticated resource allocation techniques are required in order to take advantage of the underlying architecture. At the same time, circuit aging rises as a challenging problem due to the reduction of chip dimensions, temperature, and utilization of GPU resources. Aging increases the switching delay of the transistors resulting in performance degradation, synchronization and lifetime problems. This becomes more prominent in GPUs due to the different behavior and characteristics of GPU applications. Applications utilize differently the computing resources and they consequently result in imbalanced aging. In this paper, we employ a kernel-based resource allocation for optimizing GPU throughput while simultaneously minimizing the activity divergence of Streaming Multiprocessors (SMs). The proposed methodology achieves improved throughput by effectively utilizing the characteristics of the application kernels offloaded on the platform, and reduced aging divergence among the SMs. Results show that our technique improves the GPU throughput by 18% and 13.8% for different GPU micro-architectures, while minimizing the aging divergence up to 89.6% comparing to other aging-aware methodologies.

Ballooning Graphics Memory Space in Full GPU Virtualization Environments

Advances in virtualization technology have enabled multiple virtual machines (VMs) to share resources in a physical machine (PM). With the widespread use of graphics-intensive applications, such as two-dimensional (2D) or 3D rendering, many graphics processing unit (GPU) virtualization solutions have been proposed to provide high-performance GPU services in a virtualized environment. Although elasticity is one of the major benefits in this environment, the allocation of GPU memory is still static in the sense that after the GPU memory is allocated to a VM, it is not possible to change the memory size at runtime. This causes underutilization of GPU memory or performance degradation of a GPU application due to the lack of GPU memory when an application requires a large amount of GPU memory. In this paper, we propose a GPU memory ballooning solution calledgBalloonthat dynamically adjusts the GPU memory size at runtime according to the GPU memory requirement of each VM and the GPU memory sharing overhead. ThegBalloonextends the GPU memory size of a VM by detecting performance degradation due to the lack of GPU memory. ThegBalloonalso reduces the GPU memory size when the overcommitted or underutilized GPU memory of a VM creates additional overhead for the GPU context switch or the CPU load due to GPU memory sharing among the VMs. We implemented thegBalloonby modifying thegVirt, a full GPU virtualization solution for Intel’s integrated GPUs. Benchmarking results show that thegBalloondynamically adjusts the GPU memory size at runtime, which improves the performance by up to 8% against thegVirtwith 384 MB of high global graphics memory and 32% against thegVirtwith 1024 MB of high global graphics memory.

Adaptive mesh refinement on graphics processing units for applications in gas dynamics

We present novel algorithms for cell-based adaptive mesh refinement on unstructured meshes of triangles on graphics processing units. Our implementation makes use of improved memory management techniques and a coloring algorithm for avoiding race conditions. Both the solver and AMR algorithms are entirely implemented on the GPU, with negligible communication between device and host. We show that the overhead of the AMR subroutines is small compared to the high order solver and that the proportion of total runtime spent adaptively refining the mesh decreases with the order of approximation. We apply our code to a number of benchmark problems as well as more recently proposed problems for the Euler equations that require extremely high resolution. We present the solution to a shock reflection problem that addresses the von Neumann triple point paradox with an accurately computed triple point location. Finally, we present the first solution on the full Euler equations to the problem of shock disappearance and self-similar diffraction of weak shocks around thin films.

Accelerating fuzzy cellular automata for modeling crowd dynamics

Pedestrian and crowd dynamics are physical phenomena that are fundamentally characterized by non-linear complexity. In the same time, the need of modern way of living seeks for such dynamics real time modeling also enabling computationally efficient and affordable solutions for sake of safety and easiness of people located in gathering places all over the world. Towards this direction, Cellular Automata (CAs), a parallel computational model combining macro- and microscopic inherent attributes that could severely help with adequate modeling of the aforementioned dynamics, are one of the best compromises among different competing computational techniques. In order to overcome CAs deterministic nature, in this paper the incorporation of fuzzy logic principles in a CA model that simulates crowd dynamics and crowd evacuation processes, with the usage of a Mamdani type fuzzy inference system, is proposed. More specifically, basic concepts of fuzzy logic such as linguistic variables and if-then rules are attributed to the proposed CA model to preserve fuzzy consequents and fuzzy antecedents thus resulting in a realistic and rather efficient modeling approach. Furthermore, in the paper the implementation of fuzziness in CA dynamics is tackled with the acceleration of the proposed model through fully parallel execution on Graphics Processing Units (GPU). The GPU implementation of the fuzzy CA model is analyzed in full detail and stressed against CPU corresponding implementation resulting to an important speed up of fuzzy CA execution. This is further explored through the GPU applications of the fuzzy CA model in a real building, namely the museum `CONSTANTIN XENAKIS', in Serres, Greece.