Graphics Processing Units Research Articles

Understanding the Impact of Data Center Liquid Cooling on Energy and Performance of Machine Learning and Artificial Intelligence Workloads

Abstract Traditionally, Data Centers (DC) have used air cooling for IT equipment, but as Graphics Processing Units (GPUs) evolve, they demand more power and sophisticated cooling. Aiming for efficiency, Direct Liquid Cooling (DLC) emerges as a promising solution. We evaluated the effectiveness of DLC versus traditional air cooling on a Microsoft G50 GPU server performing AI/ML tasks. The results indicated that DLC greatly enhances GPU performance, increases efficiency by 2.7% in Gflops/s, cuts power usage by 12%, reduces execution times by up to 6.22%, and lowers chip temperatures by 20° compared to air cooling. Our research develops an overall performance metric that considers data center, hardware, and chip levels, concluding that DLC is extremely beneficial for AI workloads, increasing energy savings and balancing performance with power requirements.

A Thread-Safe Lattice Boltzmann Model for Multicomponent Turbulent Jet Simulations

In this work an optimized multicomponent lattice Boltzmann (LB) model is deployed to simulate axisymmetric turbulent jets of a fluid evolving in a quiescent, immiscible environment over a wide range of dynamic regimes. The implementation of the multicomponent LB code achieves peak performance on graphic processing units (GPUs) with a significant reduction of the memory footprint, retains the algorithmic simplicity inherent to standard LB computing, and, being based on a high-order extension of the thread-safe LB algorithm, allows us to perform stable simulations at vanishingly low viscosities. The proposed approach opens attractive prospects for high-performance computing simulations of realistic turbulent flows with interfaces on GPU-based architectures.

A Survey on Design Space Exploration Approaches for Approximate Computing Systems

Approximate Computing (AxC) has emerged as a promising paradigm to enhance performance and energy efficiency by allowing a controlled trade-off between accuracy and resource consumption. It is extensively adopted across various abstraction levels, from software to architecture and circuit levels, employing diverse methodologies. The primary objective of AxC is to reduce energy consumption for executing error-resilient applications, accepting controlled and inherently acceptable output quality degradation. However, harnessing AxC poses several challenges, including identifying segments within a design amenable to approximation and selecting suitable AxC techniques to fulfill accuracy and performance criteria. This survey provides a comprehensive review of recent methodologies proposed for performing Design Space Exploration (DSE) to find the most suitable AxC techniques, focusing on both hardware and software implementations. DSE is a crucial design process where system designs are modeled, evaluated, and optimized for various extra-functional system behaviors such as performance, power consumption, energy efficiency, and accuracy. A systematic literature review was conducted to identify papers that ascribe their DSE algorithms, excluding those relying on exhaustive search methods. This survey aims to detail the state-of-the-art DSE methodologies that efficiently select AxC techniques, offering insights into their applicability across different hardware platforms and use-case domains. For this purpose, papers were categorized based on the type of search algorithm used, with Machine Learning (ML) and Evolutionary Algorithms (EAs) being the predominant approaches. Further categorization is based on the target hardware, including Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), general-purpose Central Processing Units (CPUs), and Graphics Processing Units (GPUs). A notable observation was that most studies targeted image processing applications due to their tolerance for accuracy loss. By providing an overview of techniques and methods outlined in existing literature pertaining to the DSE of AxC designs, this survey elucidates the current trends and challenges in optimizing approximate designs.

GPU Parallel Computing Architectures : Unlocking the Power of Parallelism for High-Performance Applications

Graphics Processing Units (GPUs) have evolved from specialized graphics rendering hardware to become powerful parallel computing architectures, revolutionizing high-performance computing across diverse domains. This comprehensive article explores the fundamental principles of GPU parallel computing architectures, their design, and their impact on modern computational challenges. We begin by examining the multi-core structure, memory hierarchy, and data processing capabilities of GPUs, including the SIMD execution model and thread organization. The article then delves into prominent programming models like CUDA and OpenCL, discussing their features and comparative advantages. We explore how GPUs are leveraged for general-purpose computing in scientific simulations, machine learning, and big data analytics, while also addressing the challenges inherent in GPU parallel computing, such as data transfer bottlenecks and load balancing. Recent technological advancements, including tensor cores, unified memory architecture, and ray tracing acceleration, are analyzed for their transformative potential. The article concludes by examining future directions in GPU technology, including integration with emerging technologies like quantum computing, advancements in energy efficiency, and the potential impact on solving complex global challenges. Through this comprehensive analysis, we illustrate the pivotal role of GPU parallel computing architectures in shaping the future of high-performance computing and their potential to address some of the world's most pressing computational problems.

Cluster perturbation theory. X. A parallel implementation of Lagrangian perturbation series for the coupled cluster singles and doubles ground-state energy through fifth order.

We describe an efficient implementation of cluster perturbation and Møller-Plesset Lagrangian energy series through the fifth order that targets the coupled cluster singles and doubles energy utilizing the resolution of the identity approximation. We illustrate the computational performance of the implementation by performing ground state energy calculations on systems with up to 1200 basis functions using a single node and by comparison to conventional coupled cluster singles and doubles calculations. We further show that our hybrid message passing interface/open multiprocessing parallel implementation that also utilizes graphical processing units can be used to obtain fifth order energies on systems with almost 1200 basis functions with a 90min"time to solution" running on Frontier at Oak Ridge National Laboratory.

Neural Methods for Amortized Inference

Simulation-based methods for statistical inference have evolved dramatically over the past 50 years, keeping pace with technological advancements. The field is undergoing a new revolution as it embraces the representational capacity of neural networks, optimization libraries, and graphics processing units for learning complex mappings between data and inferential targets. The resulting tools are amortized, in the sense that, after an initial setup cost, they allow rapid inference through fast feed-forward operations. In this article we review recent progress in the context of point estimation, approximate Bayesian inference, summary-statistic construction, and likelihood approximation. We also cover software and include a simple illustration to showcase the wide array of tools available for amortized inference and the benefits they offer over Markov chain Monte Carlo methods. The article concludes with an overview of relevant topics and an outlook on future research directions.

Dynamic Buffer Management in Massively Parallel Systems: The Power of Randomness

Massively parallel systems, such as Graphics Processing Units (GPUs), play an increasingly crucial role in today’s data-intensive computing. The unique challenges associated with developing system software for massively parallel hardware to support numerous parallel threads efficiently are of paramount importance. One such challenge is the design of a dynamic memory allocator to allocate memory at runtime. Traditionally, memory allocators have relied on maintaining a global data structure, such as a queue of free pages. However, in the context of massively parallel systems, accessing such global data structures can quickly become a bottleneck even with multiple queues in place. This paper presents a novel approach to dynamic memory allocation that eliminates the need for a centralized data structure. Our proposed approach revolves around letting threads employ random search procedures to locate free pages. Through mathematical proofs and extensive experiments, we demonstrate that the basic random search design achieves lower latency than the best-known existing solution, Ouroboros, in most situations. Furthermore, we develop more advanced techniques and algorithms to tackle the challenge of warp divergence and further enhance performance when free memory is limited. Building upon these advancements, our mathematical proofs and experimental results affirm that these advanced designs can yield an order of magnitude improvement over the basic design and consistently outperform the state-of-the-art by up to two orders of magnitude. To illustrate the practical implications of our work, we integrate our memory management techniques into two GPU algorithms: a hash join and a group-by. Both case studies provide compelling evidence of our approach’s pronounced performance gains.

Defending Against Deep Learning-Based Traffic Fingerprinting Attacks With Adversarial Examples

In an increasingly digital and interconnected world, online anonymity and privacy are paramount issues for Internet users. To address this, tools like The Onion Router (Tor) offer anonymous and private communication by routing traffic through multiple relays with multiple layers of encryption. However, traffic fingerprinting attacks have threatened anonymity and privacy. In response, the community has proposed additional defenses for Tor, but fingerprinting techniques that utilize deep neural networkss (DNNs) have undermined many of these defenses. The latest defenses that are both lightweight and robust against DNNs use adversarial examples, but these defenses require either the full traffic trace beforehand or a database of pre-computed adversarial examples. We propose Prism , a defense against fingerprinting attacks that utilizes adversarial examples with neither prior access to the full traffic trace nor a database. We describe a novel method of adversarial example generation as input is learned over time. Prism injects these adversarial examples into the Tor traffic stream to prevent DNNs from accurately classifying both websites and videos that a user is viewing, even if the DNN is hardened by adversarial training. We also show that the Tor network could implement Prism entirely on relays under certain conditions, extending the defense to users who may run Tor on devices without graphics processing units.

Fully integrated multi-mode optoelectronic memristor array for diversified in-sensor computing.

In-sensor computing, which integrates sensing, memory and processing functions, has shown substantial potential in artificial vision systems. However, large-scale monolithic integration of in-sensor computing based on emerging devices with complementary metal-oxide-semiconductor (CMOS) circuits remains challenging, lacking functional demonstrations at the hardware level. Here we report a fully integrated 1-kb array with 128 × 8 one-transistor one-optoelectronic memristor (OEM) cells and silicon CMOS circuits, which features configurable multi-mode functionality encompassing three different modes of electronic memristor, dynamic OEM and non-volatile OEM (NV-OEM). These modes are configured by modulating the charge density within the oxygen vacancies via synergistic optical and electrical operations, as confirmed by differential phase-contrast scanning transmission electron microscopy. Using this OEM system, three visual processing tasks are demonstrated: image sensory pre-processing with a recognition accuracy enhanced from 85.7% to 96.1% by the NV-OEM mode, more advanced object tracking with 96.1% accuracy using both dynamic OEM and NV-OEM modes and human motion recognition with a fully OEM-based in-sensor reservoir computing system achieving 91.2% accuracy. A system-level benchmark further shows that it consumes over 20 times less energy than graphics processing units. By monolithically integrating the multi-functional OEMs with Si CMOS, this work provides a cost-effective platform for diverse in-sensor computing applications.

Visual software defect prediction method based on improved recurrent criss-cross residual network

Purpose This study aims to solve the problems of large training sample size, low data sample quality, low efficiency of the currently used classical model, high computational complexity of the existing concern mechanism, and high graphics processing unit (GPU) occupancy in the current visualization software defect prediction, proposing a method for software defect prediction termed recurrent criss-cross attention for weighted activation functions of recurrent SE-ResNet (RCCA-WRSR). First, following code visualization, the activation functions of the SE-ResNet model are replaced with a weighted combination of Relu and Elu to enhance model convergence. Additionally, an SE module is added before it to filter feature information, eliminating low-weight features to generate an improved residual network model, WRSR. To focus more on contextual information and establish connections between a pixel and those not in the same cross-path, the visualized red as integer, green as integer, blue as integer images are inputted into a model incorporating a fused RCCA module for defect prediction. Design/methodology/approach Software defect prediction based on code visualization is a new software defect prediction technology, which mainly realizes the defect prediction of code by visualizing code as image, and then applying attention mechanism to extract the features of image. However, the challenges of current visualization software defect prediction mainly include the large training sample size and low sample quality of the data, and the classical models used today are not efficient, and the existing attention mechanisms have high computational complexity and high GPU occupancy. Findings Experimental evaluation using ten open-source Java data sets from PROMISE and five existing methods demonstrates that the proposed approach achieves an F-measure value of 0.637 in predicting 16 cross-version projects, representing a 6.1% improvement. Originality/value RCCA-WRSR is a new visual software defect prediction based on recurrent criss-cross attention and improved residual network. This method effectively enhances the performance of software defect prediction.

Extending GPU-accelerated Gaussian integrals in the TeraChem software package to f type orbitals: Implementation and applications.

The increasing availability of graphics processing units (GPUs) for scientific computing has prompted interest in accelerating quantum chemical calculations through their use. However, the complexity of integral kernels for high angular momentum basis functions often limits the utility of GPU implementations with large basis sets or for metal containing systems. In this work, we report the implementation of f function support in the GPU-accelerated TeraChem software package through the development of efficient kernels for the evaluation of Hamiltonian integrals. The high efficiency of the resulting code is demonstrated through density functional theory (DFT) calculations on increasingly large organic molecules and transition metal complexes, as well as coupled cluster singles and doubles calculations on water clusters. Preliminary investigations into Ni(I) catalysis with DFT and the photochemistry of MnH(CH3) with complete active space self-consistent field are also carried out. Overall, our GPU-accelerated software appears to be well-suited for fast simulation of large transition metal containing systems, as well as organic molecules.

Impact of solar-wind turbulence on a planetary bow shock. A global 3D simulation

The interaction of the solar-wind plasma with a magnetized planet generates a bow-shaped shock ahead of the wind. Over recent decades, near-Earth spacecraft observations have provided insights into the physics of the bow shock, and the findings suggest that solar-wind intrinsic turbulence influences the bow shock dynamics. On the other hand, theoretical studies, primarily based on global numerical simulations, have not yet investigated the global three-dimensional (3D) interaction between a turbulent solar wind and a planetary magnetosphere. This paper addresses this gap for the first time by presenting an investigation of the global dynamics of this interaction that provides new perspectives on the underlying physical processes. We use the newly developed numerical code Menura to examine how the turbulent nature of the solar wind influences the 3D structure and dynamics of magnetized planetary environments, such as those of Mercury, Earth, and magnetized Earth-like exoplanets. We used the hybrid particle-in-cell (PIC) code Menura to conduct 3D simulations of the turbulent solar wind and its interaction with an Earth-like magnetized planet through global numerical simulations of the magnetosphere and its surroundings. Menura runs in parallel on graphics processing units (GPUs), enabling efficient and self-consistent modeling of turbulence. By comparison with a case in which the solar wind is laminar, we show that solar-wind turbulence globally influences the shape and dynamics of the bow shock, the magnetosheath structures, and the ion foreshock dynamics. Also, a turbulent solar wind disrupts the coherence of foreshock fluctuations, induces large fluctuations on the quasi-perpendicular surface of the bow shock, facilitates the formation of bubble-like structures near the nose of the bow shock, and modifies the properties of the magnetosheath region. The turbulent nature of the solar wind impacts the 3D shape and dynamics of the bow shock, magnetosheath, and ion foreshock region. This influence should be taken into account when studying solar-wind--planet interactions in both observations and simulations. We discuss the relevance of our findings for current and future missions launched into the heliosphere.

Fast and universal single-molecule localization using multi-dimensional point spread functions

The recent development of single-molecule imaging techniques has enabled not only high accuracy spatial resolution imaging but also information rich functional imaging. Abundant information about single molecules can be encoded in its diffraction pattern and be extracted precisely (e.g. 3D position, wavelength, dipole orientation). However, sophisticated high dimensional point spread function (PSF) modeling and analyzing methods have greatly impeded the broad accessibility of these techniques. Here, we present a graphics processing unit (GPU) -based B-spline PSF modeling method that could flexibly model high dimensional PSFs with arbitrary shape without greatly increasing the model parameters. Our B-spline fitter achieves 100 times speed improvement and minimal uncertainty for each dimension, enabling efficient high dimensional single-molecule analysis. We demonstrated, both in simulations and experiments, the universality and flexibility of our B-spline fitter to accurately extract the abundant information from different types of high dimensional single-molecule data, including multicolor PSF (3D + color), multi-channel four-dimensional 4Pi-PSF (3D + interference phase) and five-dimensional vortex PSF (3D + dipole orientation).

The metaverse-industrial complex

ABSTRACT This paper explores the role that Nvidia is playing in shaping the Metaverse for industrial applications in simulation and automation. As the world's largest manufacturer of Graphics Processing Units for computationally intensive tasks and one of the largest corporations by market capitalization, Nvidia has remained relatively overlooked within critical discussions of ‘Big Tech', the Metaverse, and Artificial Intelligence. This paper draws on news media reports and publicly accessible Nvidia publications such as developer documentation and press releases to understand the role of Nvidia in shaping the Metaverse for industrial enterprises. It conceptualises the ‘Metaverse-Industrial Complex' to theorise the processes of market capture, territorialization, and platform expansionism occurring within the convergence of the Metaverse and Artificial Intelligence that is reconfiguring the governance of public and private assets and infrastructure. Through an examination of Nvidia's ‘Omniverse' platform and network of industrial partnerships, the paper examines the Metaverse-Industrial Complex as an assemblage of developer tools, platforms, data standards, and industry partnerships that converge machine learning and 3D graphics into photo-realistic ‘digital twin’ simulations of physical objects and environments.

Towards GPU-enabled serverless cloud edge platforms for accelerating HEVC video coding

Multimedia streaming has become integral to modern living, reshaping entertainment consumption, information access, and global engagement. The ascent of cloud computing, particularly serverless architectures, plays a pivotal role in this transformation, offering dynamic resource allocation, parallel execution, and automatic scaling-especially beneficial in HTTP Adaptive Streaming (HAS) applications. This study presents an event-driven serverless cloud edge platform with graphics processing units (GPUs), managed by Knative, tailored for video encoding. Two implementations of the High Efficiency Video Coding (HEVC) codec have been encapsulated in the functions: HEVC NVENC (Nvidia Encoder), that uses GPU acceleration, and x265 that only uses CPUs. Experiments focused on measuring the impact of replica requested resources on cold start, scalability and resource consumption with different allocated resources on slim and fat virtual machines (VMs). The best results are obtained when four slim replicas of the functions are deployed on a fat VM with a 8.4% reduction in encoding time for x265 and a 15.2% improvement for HEVC NVENC compared with other deployment scenarios. Comparatively, HEVC NVENC encoding is 8.3 times faster than x265. In multiresolution scenarios, GPU encoding drastically reduces segment encoding time by a factor of 12.4 between non-GPU and GPU-accelerated. These findings are important for live streaming applications where low latency is critical at all stages of the streaming process.

Edge Computing for AI-Based Brain MRI Applications: A Critical Evaluation of Real-Time Classification and Segmentation.

Medical imaging plays a pivotal role in diagnostic medicine with technologies like Magnetic Resonance Imagining (MRI), Computed Tomography (CT), Positron Emission Tomography (PET), and ultrasound scans being widely used to assist radiologists and medical experts in reaching concrete diagnosis. Given the recent massive uplift in the storage and processing capabilities of computers, and the publicly available big data, Artificial Intelligence (AI) has also started contributing to improving diagnostic radiology. Edge computing devices and handheld gadgets can serve as useful tools to process medical data in remote areas with limited network and computational resources. In this research, the capabilities of multiple platforms are evaluated for the real-time deployment of diagnostic tools. MRI classification and segmentation applications developed in previous studies are used for testing the performance using different hardware and software configurations. Cost-benefit analysis is carried out using a workstation with a NVIDIA Graphics Processing Unit (GPU), Jetson Xavier NX, Raspberry Pi 4B, and Android phone, using MATLAB, Python, and Android Studio. The mean computational times for the classification app on the PC, Jetson Xavier NX, and Raspberry Pi are 1.2074, 3.7627, and 3.4747 s, respectively. On the low-cost Android phone, this time is observed to be 0.1068 s using the Dynamic Range Quantized TFLite version of the baseline model, with slight degradation in accuracy. For the segmentation app, the times are 1.8241, 5.2641, 6.2162, and 3.2023 s, respectively, when using JPEG inputs. The Jetson Xavier NX and Android phone stand out as the best platforms due to their compact size, fast inference times, and affordability.

Digital rock physics: Calculation of effective elastic properties of heterogeneous materials using graphical processing units (GPUs)

An application based on graphical processing units (GPUs) applied to 3-D digital images is described for computing the linear anisotropic elastic properties of heterogeneous materials. The application can also retrieve the property contribution tensors of individual inclusions of any shape. The code can be executed on professional GPUs as well as on a basic laptop or personal computer Nvidia GPUs. The application is extremely fast: a calculation of the effective elastic properties of volumes consisting of about 7 million voxel elements (1913) takes less than 4 s of computational time using a single A100 GPU; 3 min for 100 million voxel elements (4793) using a single A100 GPU; 14 min for 350 million voxel elements (7033) using a single A100 GPU. Several comparisons against analytical solutions are provided. In addition, an evaluation of the anisotropic effective elastic properties of a 3-D digital image of a cracked Carrara marble sample is presented. The software can be downloaded from a permanent repository Zenodo, the link with a doi is given in the manuscript.

Propagation of Hydraulic Fractures and Natural Fractures: The Bypassing Behavior in 3D Space

Summary The interaction between natural fractures (NFs) and hydraulic fractures (HFs) in 3D space poses significant challenges to numerical simulation. The interaction behaviors between HFs and NFs in 3D space include crossing, offset, stopping, and bypassing. Many existing numerical simulators are 2D, which inherently limits their ability to account for the vertical growth of fractures. Consequently, they are unable to model the bypassing behavior effectively. In this paper, a mathematical model for the propagation of HFs and NFs in a naturally fractured reservoir is established. The displacement discontinuity method (DDM) is used to solve the rock deformation, while the finite difference method (FDM) is utilized to solve the fluid flow within fractures. The undirected graph structure is used to represent the complex fracture network, and a dynamic adjustment of grid connectivity method is implemented to describe the process by which HFs bypass NFs. By integrating with graphics processing unit (GPU) computing, an efficient 3D simulator for HFs and NFs propagation is developed. The accuracy is verified against analytical solutions and reference solutions. Subsequently, a series of numerical studies on the bypassing behavior are conducted. The primary findings are as follows: (1) The simulator can accurately capture the interactions between HFs and NFs in 3D space, including crossing, arresting, and bypassing behaviors. (2) The bypassing behavior is characterized by three distinct processes—intersection, height growth of HF, and bypassing. (3) During the intersection stage, both the injection pressure and maximum NF width increase. During the height growth stage, the pressure is relatively stable, while the maximum NF width continues to increase. These two stages together result in the “storage” phenomenon. Once the bypassing behavior occurs, both the pressure and the maximum NF width decrease sharply, leading to a “release” phenomenon. Additionally, the “storage” and “release” phenomena may impact proppant transport. (4) Given the presence of bypassing behavior, it is essential to consider the NFs in 3D fracture simulators. The simulator and its findings can provide valuable insights for field design.

The Evolution of Artificial Intelligence in Medical Imaging: From Computer Science to Machine and Deep Learning.

Artificial intelligence (AI), the wide spectrum of technologies aiming to give machines or computers the ability to perform human-like cognitive functions, began in the 1940s with the first abstract models of intelligent machines. Soon after, in the 1950s and 1960s, machine learning algorithms such as neural networks and decision trees ignited significant enthusiasm. More recent advancements include the refinement of learning algorithms, the development of convolutional neural networks to efficiently analyze images, and methods to synthesize new images. This renewed enthusiasm was also due to the increase in computational power with graphical processing units and the availability of large digital databases to be mined by neural networks. AI soon began to be applied in medicine, first through expert systems designed to support the clinician's decision and later with neural networks for the detection, classification, or segmentation of malignant lesions in medical images. A recent prospective clinical trial demonstrated the non-inferiority of AI alone compared with a double reading by two radiologists on screening mammography. Natural language processing, recurrent neural networks, transformers, and generative models have both improved the capabilities of making an automated reading of medical images and moved AI to new domains, including the text analysis of electronic health records, image self-labeling, and self-reporting. The availability of open-source and free libraries, as well as powerful computing resources, has greatly facilitated the adoption of deep learning by researchers and clinicians. Key concerns surrounding AI in healthcare include the need for clinical trials to demonstrate efficacy, the perception of AI tools as 'black boxes' that require greater interpretability and explainability, and ethical issues related to ensuring fairness and trustworthiness in AI systems. Thanks to its versatility and impressive results, AI is one of the most promising resources for frontier research and applications in medicine, in particular for oncological applications.

Tensor power flow formulations for multidimensional analyses in distribution systems

In this paper, we present two multidimensional power flow formulations based on a fixed-point iteration (FPI) algorithm to efficiently solve hundreds of thousands of Power flows (PFs) in distribution systems. The presented algorithms are the base for a new TensorPowerFlow (TPF) tool and shine for their simplicity, benefiting from multicore Central processing unit (CPU) and Graphics processing unit (GPU) parallelization. We also focus on the mathematical convergence properties of the algorithm, showing that its unique solution is at the practical operational point. The proof is validated using numerical simulations showing the robustness of the FPI algorithm compared to the classical Newton–Raphson (NR) approach. In the case study, a benchmark with different PF solution methods is performed, showing that for applications requiring a yearly simulation at 1-minute resolution, the computation time is decreased by a factor of 164, compared to the NR in its sparse formulation. Finally, a set of applications is described, highlighting the potential of the proposed formulations over a wide range of analyses in distribution systems.