Real-time GPU Research Articles

Renewable-Aware Frequency Scaling Approach for Energy-Efficient Deep Learning Clusters

Recently, renewable energy has emerged as an attractive means to reduce energy consumption costs for deep learning (DL) job processing in modern GPU-based clusters. In this paper, we propose a novel Renewable-Aware Frequency Scaling (RA-FS) approach for energy-efficient DL clusters. We have developed a real-time GPU core and memory frequency scaling method that finely tunes the training performance of DL jobs while maximizing renewable energy utilization. We introduce quantitative metrics: Deep Learning Job Requirement (DJR) and Deep Learning Job Completion per Slot (DJCS) to accurately evaluate the service quality of DL job processing. Additionally, we present a log-transformation technique to convert our non-convex optimization problem into a solvable one, ensuring the rigorous optimality of the derived solution. Through experiments involving deep neural network (DNN) model training jobs such as SqueezeNet, PreActResNet, and SEResNet on NVIDIA GPU devices like RTX3060, RTX3090, and RTX4090, we validate the superiority of our RA-FS approach. The experimental results show that our approach significantly improves performance requirement satisfaction by about 71% and renewable energy utilization by about 31% on average, compared to recent competitors.

Spark Distributed Real-Time Data and GPU Parallel Computing Based on 5G Virtual Reality

5G virtual reality has attracted many manufacturers and users with its unique immersion, interactivity and imagination characteristics, which has become the focus of new markets. The purpose of this article is to use Spark distributed real-time data system and GPU parallel computing to quickly process and analyze data. This article mainly designs a general-purpose real-time data analysis and processing system based on Spark, which mainly includes new ETL and real-time processing engine modules, and is committed to achieving higher real-time performance than traditional Hadoop. And realize fast calculation. At the same time there is universality and stability. Includes real-time flow calculations. Fast batch processing and machine learning The various types of data computers are included in this article by preparing the cutting device and adjusting the cutting output. The device is ready to effectively terminate the CUDA environment. The cudamalloc function is used to allocate a linear space of bytes to the device, and then transfer the data from the host to the device to determine the number of GPU blocks and threads. GPU parallel computing can increase the data processing speed by 27%, while the secondary programming algorithm can reduce the optimization time of the cup by 12%.

Real-Time Light Field Video Focusing and GPU Accelerated Streaming

Real-Time Light Field Video Focusing and GPU Accelerated Streaming

Simultaneous stabilization and control of the n = 1 and n = 2 resistive wall mode

DIII-D experiments demonstrate simultaneous stability measurements and control of resistive wall modes (RWMs) with toroidal mode numbers n = 1 and n = 2. RWMs with n > 1 are sometimes observed on DIII-D following the successful feedback stabilization of the n = 1 mode, motivating the development of multi-n control. A new model-based multi-mode feedback algorithm based on the VALEN physics code has been implemented on the DIII-D tokamak using a real-time GPU installed directly into the DIII-D plasma control system. In addition to stabilizing RWMs, the feedback seeks to control the stable plasma error field response, enabling compensation of the typically unaddressed DIII-D n = 2 error field component. Experiments recently demonstrated this algorithm’s ability to simultaneously control n = 1 and n = 2 perturbed fields for the first time in a tokamak, using reactor relevant external coils. Control was maintained for hundreds of wall-times above the n = 1 no-wall pressure limit and approaching the n = 1 and n = 2 ideal-wall limits. Furthermore, a rotating non-zero target was set for the feedback, allowing stability to be assessed by monitoring the rotating plasma response (PR) while maintaining control. This novel technique can be viewed as a closed-loop extension of active MHD spectroscopy, which has been used to validate stability models through comparisons of the PR to applied, open-loop perturbations. The closed-loop response measurements are consistent with open-loop MHD spectroscopy data over a wide range of β N approaching the n = 1 ideal-wall limit. These PR measurements were then fit to produce both VALEN and single-mode stability models. These models allowed for important plasma stability information to be determined and have been shown to agree with experimentally observed RWM growth rates.

Real-Time Visualization of Large-Scale Geological Models With Nonlinear Feature-Preserving Levels of Detail.

The rapidly growing size and complexity of 3D geological models has increased the need for level-of-detail techniques and compact encodings to facilitate interactive visualization. For large-scale hexahedral meshes, state-of-the-art approaches often employ wavelet schemes for level of detail as well as for data compression. Here, wavelet transforms serve two purposes: (1) they achieve substantial compression for data reduction; and (2) the multiresolution encoding provides levels of detail for visualization. However, in coarser detail levels, important geometric features, such as geological faults, often get too smoothed out or lost, due to linear translation-invariant filtering. The same is true for attribute features, such as discontinuities in porosity or permeability. We present a novel, integrated approach addressing both purposes above, while preserving critical data features of both model geometry and its attributes. Our first major contribution is that we completely decouple the computation of levels of detail from data compression, and perform nonlinear filtering in a high-dimensional data space jointly representing the geological model geometry with its attributes. Computing detail levels in this space enables us to jointly preserve features in both geometry and attributes. While designed in a general way, our framework specifically employs joint bilateral filters, computed efficiently on a high-dimensional permutohedral grid. For data compression, after the computation of all detail levels, each level is separately encoded with a standard wavelet transform. Our second major contribution is a compact GPU data structure for the encoded mesh and attributes that enables direct real-time GPU visualization without prior decoding.

TBEM: Testing-Based GPU-Memory Consumption Estimation for Deep Learning

Deep Learning (DL) has been successfully implemented and deployed to various software service applications. During the training process of DL, a large amount of GPU computing resources is required, but it is difficult for developers to accurately calculate the GPU resources that the model may consume before running, which brings great inconvenience to the development of DL systems. Especially in today’s cloud-based model training. Therefore, it is very important to estimate the GPU memory resources that the DL model may use in a certain computing framework. Existing work has focused on static analysis methods to assess GPU memory consumption, highly coupled with the framework, and lack of research on low-coupled GPU memory consumption of the framework. In this article, we propose TBEM, which is a test-based method for estimating the memory usage of the DL model. First, TBEM generates enough DL models using an orthogonal array testing strategy and a classical neural network design pattern. Then, TBEM generates DL model tested in a real environment to obtain the real-time GPU memory usage values corresponding to the model. After obtaining the data of different models and corresponding GPU usage values, the data is analyzed by regression.

Real-Time Nonrigid Mosaicking of Laparoscopy Images.

The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D non-rigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on in vivo and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.

Dynamic Memory Bandwidth Allocation for Real-Time GPU-Based SoC Platforms

Heterogeneous SoC platforms, comprising both general purpose CPUs and accelerators, such as a GPU, are becoming increasingly attractive for real-time and mixed-criticality systems to cope with the computational demand of data parallel applications. However, contention for access to shared main memory can lead to significant performance degradation on both CPU and GPU. Existing work has shown that memory bandwidth throttling is effective in protecting real-time applications from memory-intensive, best-effort (BE) ones; however, due to the inherent pessimism involved in worst-case execution time (WCET) estimation, such approaches can unduly restrict the bandwidth available to BE applications. In this article, we propose a novel memory bandwidth allocation scheme where we dynamically monitor the progress of a real-time application and increase the bandwidth share of BE ones whenever it is safe to do so. Specifically, we demonstrate our approach by protecting a real-time GPU kernel from BE CPU tasks. Based on profiling information, we first build a WCET estimation model for the GPU kernel. Using such model, we then show how to dynamically recompute on-line the maximum memory budget that can be allocated to BE tasks without exceeding the kernel’s assigned execution budget. We implement our proposed technique on NVIDIA embedded SoC and demonstrate its effectiveness on a variety of GPU and CPU benchmarks.

BCC 볼륨 데이터로부터 실시간으로 메시 형태의 등가면을 추출하는 GPU 기법

BCC 볼륨 데이터로부터 실시간으로 메시 형태의 등가면을 추출하는 GPU 기법

Efficient Algorithms for Real-Time GPU Volumetric Cloud Rendering with Enhanced Geometry

This paper presents several new techniques for volumetric cloud rendering using efficient algorithms and data structures based on ray-tracing methods for cumulus generation, achieving an optimum balance between realism and performance. These techniques target applications such as flight simulations, computer games, and educational software, even with conventional graphics hardware. The contours of clouds are defined by implicit mathematical expressions or triangulated structures inside which volumetric rendering is performed. Novel techniques are used to reproduce the asymmetrical nature of clouds and the effects of light-scattering, with low computing costs. The work includes a new method to create randomized fractal clouds using a recursive grammar. The graphical results are comparable to those produced by state-of-the-art, hyper-realistic algorithms. These methods provide real-time performance, and are superior to particle-based systems. These outcomes suggest that our methods offer a good balance between realism and performance, and are suitable for use in the standard graphics industry.

A memory-driven scheduling scheme and optimization for concurrent execution in GPU

Concurrent execution of GPU tasks is available in modern GPU device. However, limited device memory is an obvious bottleneck in executing many GPU tasks. And the task priority and system performance are often ignored. To address these, a real-time GPU scheduling scheme is proposed in this paper. A reservation algorithm based on device memory(RBDM) is adopted to provide more opportunity for the High-priority task in the scheme. high priority first wake (HPFW) and small memory HPFW (SM-HPFW) are employed in the scheduling of waiting tasks to improve the priority response time and system performance. A CPU-based monitor is developed to check the GPU task execution. Experiments show the RBDM can work effectively. Compared with FIFO, HPFW can decrease overall priority response time significantly. Overall task completion time can be reduced by 20 % using the SM-HPFW while the distribution of device memory requirement of GPU tasks is even.

NaNet: a configurable NIC bridging the gap between HPC and real-time HEP GPU computing

NaNet is a FPGA-based PCIe Network Interface Card (NIC) design with GPUDirect and Remote Direct Memory Access (RDMA) capabilities featuring a configurable and extensible set of network channels. The design currently supports both standard—Gbe (1000BASE-T) and 10GbE (10Base-R)—and custom—34 Gbps APElink and 2.5 Gbps deterministic latency KM3link—channels, but its modularity allows for straightforward inclusion of other link technologies. The GPUDirect feature combined with a transport layer offload module and a data stream processing stage makes NaNet a low-latency NIC suitable for real-time GPU processing. In this paper we describe the NaNet architecture and its performances, exhibiting two of its use cases: the GPU-based low-level trigger for the RICH detector in the NA62 experiment at CERN and the on-/off-shore data transport system for the KM3NeT-IT underwater neutrino telescope.

Fractional Reyes-Style Adaptive Tessellation for Continuous Level of Detail

In this paper we present a fractional parametric splitting scheme for Reyes-style adaptive tessellation. Our parallel algorithm generates crack-free tessellation from a parametric surface, which is also free of sudden temporal changes under animation. Continuous level of detail is not addressed by existing Reyes-style methods, since these aim to produce subpixel-sized micropolygons, where topology changes are no longer noticeable. Using our method, rendering pipelines that use larger triangles, thus sensitive to geometric popping, may also benefit from the quality of the split-dice tessellation stages of Reyes. We demonstrate results on a real-time GPU implementation, going beyond the limited quality and resolution of the hardware tessellation unit. In contrast to previous split-dice methods, our split stage is compatible with the fractional hardware tessellation scheme that has been designed for continuous level of detail.

A digital hardware design for real-time simulation of large neural-system models in physical settings

The organization of neural systems reflects the specific complexities of the physical environments in which they operate. In order to address this relationship more directly, there is increasing interest in testing real-time neural simulations that interface with the physical world. We describe a new simulation approach that allows us to run large, sophisticated neural models on low-power embedded commodity hardware such as field-programmable gate arrays (FPGAs). A custom digital circuit was designed to approximate the collective outputs of populations of neurons that have correlated activity. These populations are taken to represent physical quantities in their spike rates. Information processing (e.g. function approximation) is taken to be determined by synaptic weights. This design is based on the Neural Engineering Framework (NEF), which bridges the gap between neural activity and higher-level behaviour [1,2]. Populations are grouped together on hardware execution components, which we call “population units”, that perform time-multiplexing in order to simulate 1024 populations per timestep. The population unit represents each population as a weighted sum of principal components of the neural tuning curves summed with a model of the associated high-frequency spike-related fluctuations. These principal components span the functions that weighted sums of spikes can approximate without being dominated by spike-related noise. Populations running on the same population unit use the same principal components, which saves memory and improves the speed of the simulation. Clustering is performed prior to simulation, which groups together populations which can be accurately represented by shared principal components. The hardware does not need to be customized or regenerated in order to simulate different networks. It can be programmed with a network description generated by a compiler that operates as a backend to the Nengo simulator. (Nengo was used to run the Spaun model [2].) The design was implemented on an FPGA and was able to run simulations of up to 45 thousand populations of neurons (a realistic surrogate model of about 1-5 million point neurons) in real-time at 12-bit accuracy on a 1 millisecond timestep. Input and output to the hardware is over Gigabit Ethernet and can be collected from a PC running Nengo for simulation control and visualization. This implementation allows real-time approximate simulation of about the same scale as the largest real-time GPU simulations in Nengo, but using much less power. Furthermore, the FPGA chip is suitable for embedded applications such as mobile robots, cameras, etc. This work greatly facilitates simulation of an essential feature of neural systems, their embodiment and interaction with the physical world.

Real-time finite difference-based sound synthesis using graphics processors

Finite difference methods can be used to model the vibrations of plates and membranes; the output of these numerical simulations can then be used to generate realistic and dynamic sounds. To create interactive, playable software instruments with finite difference methods, we need to be able run large simulations in real-time. Real-time finite difference-based simulations of large models are typically too compute-intensive to run on CPUs. The ubiquity of graphics processors (GPUs) today make them obvious choices for speeding up such applications. We have implemented finite difference simulations that run in real-time on GPUs. We will describe how we address the problems that arise from interactions between real-time audio constraints and GPU architecture and performance characteristics, and demonstrate the current version of FDS, our Finite Difference Synthesizer.

Simulation and Data-Processing Framework for Hybrid Synthetic Aperture THz Systems Including THz-Scattering

We present a simulation and data-processing framework that incorporates full 3-D configuration, simulation, and reconstruction functionalities for hybrid synthetic aperture THz systems, which combine mechanically scanned real imaging projection (1-D) and synthetic imaging reconstruction (2-D). The simulation is based on a ray-casting approach including models for reflections at rough surfaces and angular antenna sensitivity. A near real-time GPU reconstruction is introduced to process large amounts of data in parallel. Techniques for fast fusion of reconstructed data from several acquisition directions are presented. They serve as basis to visualize a fully consistent 3-D object. The modular framework allows a fast, easy, and efficient evaluation and prediction of system properties including scattering behavior for these types of scanner setups.

Racking focus and tracking focus on live video streams: a stereo solution

The ability to produce dynamic Depth of Field effects in live video streams was until recently a quality unique to movie cameras. In this paper, we present a computational camera solution coupled with real-time GPU processing to produce runtime dynamic Depth of Field effects. We first construct a hybrid-resolution stereo camera with a high-res/low-res camera pair. We recover a low-res disparity map of the scene using GPU-based Belief Propagation, and subsequently upsample it via fast Cross/Joint Bilateral Upsampling. With the recovered high-resolution disparity map, we warp the high-resolution video stream to nearby viewpoints to synthesize a light field toward the scene. We exploit parallel processing and atomic operations on the GPU to resolve visibility when multiple pixels warp to the same image location. Finally, we generate racking focus and tracking focus effects from the synthesized light field rendering. All processing stages are mapped onto NVIDIA's CUDA architecture. Our system can produce racking and tracking focus effects for the resolution of 640×480 at 15 fps.

Real-time rendering of approximate caustics under environment illumination

We present a real-time GPU caustics rendering technique for dynamic scenes under environment illumination taking into account light occlusion. The dynamic scenes consist of caustic objects (reflective and/or refractive objects which produce caustics) and receiver objects (non-reflective and non-refractive), that can be translated and rotated. As the light source, we consider environment illumination (distant lights from all directions) which we approximate as a set of important directional lights. Our rendering technique is able to generate approximate caustics (cast on receiver objects) as well as volumetric caustics. As the preprocessing, we precompute the caustic patterns of caustic objects for several directional lights and store them in caustic images. During the rendering, we interpolate the precomputed caustic patterns based on the important directional lights which approximate the given environment illumination. The important directional lights are obtained by using our proposed environment cube map segmentation technique. Our proposed technique is able to generate real-time caustics which are visually similar to the caustic generated by using a commercial renderer mental ray.

F5-4 SPH法による頭部衝突解析とGPUを用いた実時間可視化(F-5 構造物の健全性評価と計算力学技術,フォーラム)

F5-4 SPH法による頭部衝突解析とGPUを用いた実時間可視化(F-5 構造物の健全性評価と計算力学技術,フォーラム)

Real-time soft shadows in dynamic scenes using spherical harmonic exponentiation

Previous methods for soft shadows numerically integrate over many light directions at each receiver point, testing blocker visibility in each direction. We introduce a method for real-time soft shadows in dynamic scenes illuminated by large, low-frequency light sources where such integration is impractical. Our method operates on vectors representing low-frequency visibility of blockers in the spherical harmonic basis. Blocking geometry is modeled as a set of spheres; relatively few spheres capture the low-frequency blocking effect of complicated geometry. At each receiver point, we compute the product of visibility vectors for these blocker spheres as seen from the point. Instead of computing an expensive SH product per blocker as in previous work, we perform inexpensive vector sums to accumulate the log of blocker visibility. SH exponentiation then yields the product visibility vector over all blockers. We show how the SH exponentiation required can be approximated accurately and efficiently for low-order SH, accelerating previous CPU-based methods by a factor of 10 or more, depending on blocker complexity, and allowing real-time GPU implementation.