High-end Graphics Processing Units Research Articles

Accelerating Neural Network Inference With Processing-in-DRAM: From the Edge to the Cloud

Neural networks (NNs) are growing in importance and complexity. An NN’s performance (and energy efficiency) can be bound either by computation or memory resources. The processing-in-memory (PIM) paradigm, where computation is placed near or within memory arrays, is a viable solution to accelerate memory-bound NNs. However, PIM architectures vary in form, where different PIM approaches lead to different tradeoffs. Our goal is to analyze, discuss, and contrast dynamic random-access memory (DRAM)-based PIM architectures for NN performance and energy efficiency. To do so, we analyze three state-of-the-art PIM architectures: 1) UPMEM, which integrates processors and DRAM arrays into a single 2-D chip, 2) Mensa, a 3-D-stacking-based PIM architecture tailored for edge devices, and 3) SIMDRAM, which uses the analog principles of DRAM to execute bit-serial operations. Our analysis reveals that PIM greatly benefits memory-bound NNs: 1) UPMEM provides 23× the performance of a high-end graphics processing unit (GPU) when the GPU requires memory oversubscription for a general matrix–vector multiplication kernel, 2) Mensa improves energy efficiency and throughput by 3.0× and 3.1× over the baseline Edge tensor processing unit for 24 Google edge NN models, and 3) SIMDRAM outperforms a central processing unit/graphics processing unit by 16.7×/1.4× for three binary NNs. We conclude that the ideal PIM architecture for NN models depends on a model's distinct attributes, due to the inherent architectural design choices.

Real-Time Driving Scene Understanding via Efficient 3-D LIDAR Processing

Three-dimensional light detection and ranging (3D LIDAR) sensors are widely used in autonomous vehicles; however, its drawback is the significant computation-processing requirement. Although parallel processing using high-end graphics processing units (GPUs) can solve the aforementioned problem, the cost of high-end GPUs prevents 3D LIDAR from being employed in commercial vehicles. In this study, a new fast and efficient perception method using 3D LIDAR is developed. The aim is to develop a real-time perception method that can run on both a low-end GPU and a central processing unit (CPU). The proposed method consists of three parts: preprocessing, point cloud classification network (PCCN), and filtering. To process a point cloud using minimal computation and achieve good performance, handcrafted features and learning-based features are used in the PCCN—the former capture the global information of the given point cloud, whereas the latter encode the local information of the points in the cloud. Finally, the proposed LIDAR perception method is applied to the SemanticKITTI dataset, and it is validated by comparing with other competing methods in terms of the accuracy and computation time on the CPU.

Foveated light-field display and real-time rendering for virtual reality.

Glasses-free light field displays have significantly progressed due to advances in high-resolution microdisplays and high-end graphics processing units (GPUs). However, for near-eye light-field displays requiring portability, the fundamental trade-off regarding achieved spatial resolution remains: retinal blur quality must be degraded; otherwise, computational consumption increases. This has prevented synthesizing the high-quality light field from being fast. By integrating off-the-shelf gaze tracking modules into near-eye light-field displays, we present wearable virtual reality prototypes supporting human visual system-oriented focus cues. An optimized, foveated light field is delivered to each eye subject to the gaze point, providing more natural visual experiences than state-of-the-art solutions. Importantly, the factorization runtime can be immensely reduced, since the image resolution is only high within the gaze cone. In addition, we demonstrate significant improvements in computation and retinal blur quality over counterpart near-eye displays.

Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software

The use of mixed precision in numerical algorithms is a promising strategy for accelerating scientific applications. In particular, the adoption of specialized hardware and data formats for low-precision arithmetic in high-end GPUs (graphics processing units) has motivated numerous efforts aiming at carefully reducing the working precision in order to speed up the computations. For algorithms whose performance is bound by the memory bandwidth, the idea of compressing its data before (and after) memory accesses has received considerable attention. One idea is to store an approximate operator–like a preconditioner–in lower than working precision hopefully without impacting the algorithm output. We realize the first high-performance implementation of an adaptive precision block-Jacobi preconditioner which selects the precision format used to store the preconditioner data on-the-fly, taking into account the numerical properties of the individual preconditioner blocks. We implement the adaptive block-Jacobi preconditioner as production-ready functionality in the Ginkgo linear algebra library, considering not only the precision formats that are part of the IEEE standard, but also customized formats which optimize the length of the exponent and significand to the characteristics of the preconditioner blocks. Experiments run on a state-of-the-art GPU accelerator show that our implementation offers attractive runtime savings.

Agglomerative Memory and Thread Scheduling for High-Performance Ray-Tracing on GPUs

Ray-tracing rendering has long been considered as a promising technology to enable a higher level of visual experience. The democratization of the ray-tracing rendering to consumer platforms, however, poses significant challenges to rendering hardware and software due to its highly irregular computing patterns. In fact, modern ray-tracing techniques typically depend on a tree-based acceleration structure to reduce the computing complexity of intersection testing of rays and graphics primitives. The traversal by a massive number of rays on a graphics processing unit (GPU) incurs a significant amount of irregular memory traffic, which turns out to be a major stumbling block for real-time performance. In this work, a scheduling mechanism, so-called Agglomerative Memory and Thread Scheduling, is proposed to unleash the inherence parallelism in the ray-tracing process on GPUs. It is associated with a tile-based ray-tracing framework in which the acceleration structure (i.e., KD-tree in this work) is partitioned into subtrees that can be completely loaded into the on-chip L1 cache inside a streaming multiprocessor. An effective scheduling mechanism collects threads with regard to the subtrees hit by their respective rays and regroup threads into warps for dispatching. In addition, subtrees are dynamically preloaded into the L1 cache of multiprocessors in an on-demand fashion. The proposed scheduler can be integrated on today’s high-end GPUs with only minor overhead. Microarchitecture simulation results prove that the proposed framework significantly improves memory efficiency and outperforms a traditional GPU microarchitecture by 47.4% for average.

SODECL

Stochastic differential equations (SDEs) are widely used to model systems affected by random processes. In general, the analysis of an SDE model requires numerical solutions to be generated many times over multiple parameter combinations. However, this process often requires considerable computational resources to be practicable. Due to the embarrassingly parallel nature of the task, devices such as multi-core processors and graphics processing units (GPUs) can be employed for acceleration. Here, we present SODECL (https://github.com/avramidis/sodecl), a software library that utilizes such devices to calculate multiple orbits of an SDE model. To evaluate the acceleration provided by SODECL, we compared the time required to calculate multiple orbits of an exemplar stochastic model when one CPU core is used, to the time required when using all CPU cores or a GPU. In addition, to assess scalability, we investigated how model size affected execution time on different parallel compute devices. Our results show that when using all 32 CPU cores of a high-end high-performance computing node, the task is accelerated by a factor of up to ≈6.7, compared to when using a single CPU core. Executing the task on a high-end GPU yielded accelerations of up to ≈4.5, compared to a single CPU core.

Cluster-Based and GPU-Driven Parallel Computing Model to Accelerate Circuit Simulation

In this digital age circuit design, analysis and validation is not only fundamental step but quite crucial in all the industries and in research. Simulation software is available for circuit analysis, but they all prove to be slower for very large circuit simulation or to execute thousands of iteration of transient analysis. Accelerating simulator is as important as speeding up circuit design. In this paper we have addressed circuit analysis using parallel computing approach on Graphics Processing Unit (GPU). Now a day’s high end GPUs are available with sufficient memory in the architecture itself. Circuit processing functions are analysed to search compute intensive functions. Mathematical operations are redesigned so that it will execute in parallel. LU decomposition algorithm and complex math operations are converted in parallel form. Some mathematical operations are simplified to merge them in suitable cluster. Clustering approach is used which helps in finding kernel of uniform operations to map on GPU cores. GPU programming strategies like if-else in-lining, parallel reduction etc are useful in accelerating circuit operations. Use of look up tables in shared memory or constant memory proves to be useful in fast data access. At least 15% speed gain is achieved for operational analysis and 40% for transient analysis of regular circuits.

GPU Computing with Python: Performance, Energy Efficiency and Usability

In this work, we examine the performance, energy efficiency, and usability when using Python for developing high-performance computing codes running on the graphics processing unit (GPU). We investigate the portability of performance and energy efficiency between Compute Unified Device Architecture (CUDA) and Open Compute Language (OpenCL); between GPU generations; and between low-end, mid-range, and high-end GPUs. Our findings showed that the impact of using Python is negligible for our applications, and furthermore, CUDA and OpenCL applications tuned to an equivalent level can in many cases obtain the same computational performance. Our experiments showed that performance in general varies more between different GPUs than between using CUDA and OpenCL. We also show that tuning for performance is a good way of tuning for energy efficiency, but that specific tuning is needed to obtain optimal energy efficiency.

Simulation of Nonspherical Asteroid Landers: Contact Modeling and Shape Effects on Bouncing

This paper presents a methodology for the fast, parallelized simulation of the bouncing deployment of lander/rover probe to an asteroid or comet. The target asteroid or comet is represented implicitly with a signed distance field, allowing for fast collision detection with high-resolution shape models. The gravity field of the target body is voxelized to make run-time computations efficient. Collisions and contact motion between a probe with arbitrary shape/inertia and the target body are evaluated using a hard-surface contact model with restitution and friction. The shape, gravity, and contact models are well suited for parallel implementation on a high-end graphics processing unit found in modern personal computers, which enables broad but detailed studies with a relatively large amount of simulations. Probe deployment to asteroid Itokawa is simulated to examine the trends in the motion of different probe shapes. Similarly, the effect of variations in the surface interaction coefficients and probe mass distribution are identified. These results are compared and contrasted with previously observed trends in the motion of strictly spherical probes and have implications for the general design of small-body surface exploration probes.

A 34-FPS 698-GOP/s/W Binarized Deep Neural Network-Based Natural Scene Text Interpretation Accelerator for Mobile Edge Computing

The scene text interpretation is a critical part of the natural scene interpretation. Currently, most of the existing work is based on high-end graphics processing units (GPUs) implementation, which is commonly used on the server side. However, in Internet of Things (IoT) application scenarios, the communication overhead from the edge device to the server is quite large, which sometimes even dominates the total processing time. Hence, the edge-computing oriented design is needed to solve this problem. In this paper, we present an architectural design and implementation of a natural scene text interpretation (NSTI) accelerator, which can classify and localize the text region on pixel-level efficiently in real-time on mobile devices. To target the real-time and low-latency processing, the binary convolutional encoder–decoder network is adopted as the core architecture to enable massive parallelism due to its binary feature. Massively parallelized computations and a highly pipelined data flow control enhance its latency and throughput performance. In addition, all the binarized intermediate results and parameters are stored on chip to eliminate the power consumption and latency overhead of the off-chip communication. The NSTI accelerator is implemented in a 40 nm CMOS technology, which can process scene text images (size of 128 × 32) at 34 fps and latency of 40 ms for pixelwise interpretation with the pixelwise classification accuracy over 90% on ICDAR-03 and ICDAR-13 dataset. The real energy-efficiency is 698 GOP/s/W and the peak energy-efficiency can get up to 7825 GOP/s/W. The proposed accelerator is 7 times more energy efficient than its optimized GPU-based implementation counterpart, while maintaining a real-time throughput with latency of 40 ms.

Heterogeneous computing for epidemiological model fitting and simulation

BackgroundOver the last years, substantial effort has been put into enhancing our arsenal in fighting epidemics from both technological and theoretical perspectives with scientists from different fields teaming up for rapid assessment of potentially urgent situations. This paper focusses on the computational aspects of infectious disease models and applies commonly available graphics processing units (GPUs) for the simulation of these models. However, fully utilizing the resources of both CPUs and GPUs requires a carefully balanced heterogeneous approach.ResultsThe contribution of this paper is twofold. First, an efficient GPU implementation for evaluating a small-scale ODE model; here, the basic S(usceptible)-I(nfected)-R(ecovered) model, is discussed. Second, an asynchronous particle swarm optimization (PSO) implementation is proposed where batches of particles are sent asynchronously from the host (CPU) to the GPU for evaluation. The ultimate goal is to infer model parameters that enable the model to correctly describe observed data. The particles of the PSO algorithm are candidate parameters of the model; finding the right one is a matter of optimizing the likelihood function which quantifies how well the model describes the observed data. By employing a heterogeneous approach, in which both CPU and GPU are kept busy with useful work, speedups of 10 to 12 times can be achieved on a moderate machine with a high-end consumer GPU as compared to a high-end system with 32 CPU cores.ConclusionsUtilizing GPUs for parameter inference can bring considerable increases in performance using average host systems with high-end consumer GPUs. Future studies should evaluate the benefit of using newer CPU and GPU architectures as well as applying this method to more complex epidemiological scenarios.

Rapid data processing for ultrafast X-ray computed tomography using scalable and modular CUDA based pipelines

Ultrafast X-ray tomography is an advanced imaging technique for the study of dynamic processes basing on the principles of electron beam scanning. A typical application case for this technique is e.g. the study of multiphase flows, that is, flows of mixtures of substances such as gas–liquidflows in pipelines or chemical reactors. At Helmholtz-Zentrum Dresden-Rossendorf (HZDR) a number of such tomography scanners are operated. Currently, there are two main points limiting their application in some fields. First, after each CT scan sequence the data of the radiation detector must be downloaded from the scanner to a data processing machine. Second, the current data processing is comparably time-consuming compared to the CT scan sequence interval. To enable online observations or use this technique to control actuators in real-time, a modular and scalable data processing tool has been developed, consisting of user-definable stages working independently together in a so called data processing pipeline, that keeps up with the CT scanner’s maximal frame rate of up to 8 kHz. The newly developed data processing stages are freely programmable and combinable. In order to achieve the highest processing performance all relevant data processing steps, which are required for a standard slice image reconstruction, were individually implemented in separate stages using Graphics Processing Units (GPUs) and NVIDIA’s CUDA programming language. Data processing performance tests on different high-end GPUs (Tesla K20c, GeForce GTX 1080, Tesla P100) showed excellent performance. Program summary/new version program summaryProgram Title: GLADOS/RISAProgram Files doi:http://dx.doi.org/10.17632/65sx747rvm.1Licensing provisions: LGPLv3Programming language: C++/CUDASupplementary material: Test data set, used for the performance analysis.Nature of problem: Ultrafast computed tomography is performed with a scan rate of up to 8 kHz. To obtain cross-sectional images from projection data computer-based image reconstruction algorithms must be applied. The objective of the presented program is to reconstruct a data stream of around 1.3 GB s−1 in a minimum time period. Thus, the program allows to go into new fields of application and to use in the future even more compute-intensive algorithms, especially for data post-processing, to improve the quality of data analysis.Solution method: The program solves the given problem using a two-step process: first, by a generic, expandable and widely applicable template library implementing the streaming paradigm (GLADOS); second, by optimized processing stages for ultrafast computed tomography implementing the required algorithms in a performance-oriented way using CUDA (RISA). Thereby, task-parallelism between the processing stages as well as data parallelism within one processing stage is realized.

Optimizing the Monte Carlo Neutron Cross-Section Construction Code XSBench for MIC and GPU Platforms

XSBench is a proxy application used to study the performance of nuclear macroscopic cross-section data construction, which is usually the most time-consuming process in Monte Carlo neutron transport simulations. In this technical note we report on our experience in optimizing XSBench to Intel multicore central processing units (CPUs), many integrated core coprocessors (MICs), and Nvidia graphics processing units (GPUs). The continuous-energy cross-section construction in the Monte Carlo simulation of the Hoogenboom-Martin large problem is used in our benchmark. We demonstrate that through several tuning techniques, particularly data prefetch, the performance of XSBench on each platform can be desirably improved compared to the original implementation on the same platform. It is shown that the performance gain is 1.46× on the Westmere CPU, 1.51× on the Haswell CPU, 2.25× on the Knights Corner (KNC) MIC, and 5.98× on the Kepler GPU. The comparison across different platforms shows that when using the high-end Haswell CPU as the baseline, the KNC MIC is 1.63× faster while the high-end Kepler GPU is 2.20× faster.

Embedded-Based Graphics Processing Unit Cluster Platform for Multiple Sequence Alignments.

High-end graphics processing units (GPUs), such as NVIDIA Tesla/Fermi/Kepler series cards with thousands of cores per chip, are widely applied to high-performance computing fields in a decade. These desktop GPU cards should be installed in personal computers/servers with desktop CPUs, and the cost and power consumption of constructing a GPU cluster platform are very high. In recent years, NVIDIA releases an embedded board, called Jetson Tegra K1 (TK1), which contains 4 ARM Cortex-A15 CPUs and 192 Compute Unified Device Architecture cores (belong to Kepler GPUs). Jetson Tegra K1 has several advantages, such as the low cost, low power consumption, and high applicability, and it has been applied into several specific applications. In our previous work, a bioinformatics platform with a single TK1 (STK platform) was constructed, and this previous work is also used to prove that the Web and mobile services can be implemented in the STK platform with a good cost-performance ratio by comparing a STK platform with the desktop CPU and GPU. In this work, an embedded-based GPU cluster platform will be constructed with multiple TK1s (MTK platform). Complex system installation and setup are necessary procedures at first. Then, 2 job assignment modes are designed for the MTK platform to provide services for users. Finally, ClustalW v2.0.11 and ClustalWtk will be ported to the MTK platform. The experimental results showed that the speedup ratios achieved 5.5 and 4.8 times for ClustalW v2.0.11 and ClustalWtk, respectively, by comparing 6 TK1s with a single TK1. The MTK platform is proven to be useful for multiple sequence alignments.

IP Address Lookup by Using GPU

We present a novel parallel IP address lookup architecture based on graphics processing unit (GPU) via compute unified device architecture (CUDA). Our architecture consists of two functions: 1) host function and 2) device function. The host function is executed by a CPU to construct and update the data structure of IP address lookup executed by the device function in a GPU. Both host and device functions are executed simultaneously to fully utilize computational resources. To shorten the lookup time, a trie-based data structure optimized for CUDA is developed. The trie-based data structure uses multi-bit stride to shorten the trie depth and also improves the efficiency of texture cache in GPUs. The experimental results show that a low-end G92 GPU can achieve a throughput of more than 1.3 billion packets per second for IPv4 routing tables with more than 350K prefixes while a high-end GT200 GPU can further double the performance. By employing dual data structures, the implementation can support several hundred thousand updates per second.

Constructing a Bioinformatics Platform with Web and Mobile Services Based on NVIDIA Jetson TK1

Current high-end graphics processing units (abbreviate to GPUs), such as NVIDIA Tesla, Fermi, Kepler series cards which contain up to thousand cores per-chip, are widely used in the high performance computing fields. These GPU cards (called desktop GPUs) should be installed in personal computers/servers with desktop CPUs; moreover, the cost and power consumption of constructing a high performance computing platform with these desktop CPUs and GPUs are high. NVIDIA releases Tegra K1, called Jetson TK1, which contains 4 ARM Cortex-A15 CPUs and 192 CUDA cores (Kepler GPU) and is an embedded board with low cost, low power consumption and high applicability advantages for embedded applications. NVIDIA Jetson TK1 becomes a new research direction. Hence, in this paper, a bioinformatics platform was constructed based on NVIDIA Jetson TK1. ClustalWtk and MCCtk tools for sequence alignment and compound comparison were designed on this platform, respectively. Moreover, the web and mobile services for these two tools with user friendly interfaces also were provided. The experimental results showed that the cost-performance ratio by NVIDIA Jetson TK1 is higher than that by Intel XEON E5-2650 CPU and NVIDIA Tesla K20m GPU card.

A scalable architecture for geometric correction of multi-projector display systems

Multi-projector displays allow the realization of large and immersive projection environments by allowing the tiling of projections from multiple projectors. Such tiled displays require real time geometrical warping of the content that is being projected from each projector. This geometrical warping is a computationally intensive operation and is typically applied using high-end graphics processing units (GPUs) that are able to process a defined number of projector channels. Furthermore, this limits the applicability of such multi-projector display systems only to the content that is being generated using desktop based systems. In this paper we propose a platform independent FPGA based scalable hardware architecture for geometric correction of projected content that allows addition of each projector channel at a fractional increase in logic area. The proposed scheme provides real time correction of HD quality video streams and thus enables the use of this technology for embedded and standalone devices.

Evaluation and Acceleration of High-Throughput Fixed-Point Object Detection on FPGAs

Reliance on object or people detection is rapidly growing beyond surveillance to industrial and social applications. The histogram of oriented gradients (HOG), one of the most popular object detection algorithms, achieves high detection accuracy but delivers just under 1 frame/s on a high-end CPU. Field-programmable gate array (FPGA) accelerations of this algorithm are limited by the intensive floating-point computations. All current fixed-point HOG implementations use large bit width to maintain detection accuracy, or perform poorly at reduced data precision. In this paper, we introduce the full-image evaluation methodology to explore the FPGA implementation of HOG using reduced bit width. This approach lessens the required area resources on the FPGA, and increases the clock frequency and hence the throughput per device through increased parallelism. We evaluate the detection accuracy of the fixed-point HOG by applying state-of-the-art computer vision pedestrian detection evaluation metrics and show it performs as well as the original floating-point code from OpenCV. We then show our single FPGA implementation achieves a $68.7 \times $ higher throughput than a high-end CPU, $5.1 \times $ higher than a high-end graphics processing unit (GPU), and $7.8 \times $ higher than the same implementation using floating-point on the same FPGA. A power consumption comparison for different platforms shows our fixed-point FPGA implementation uses $130 \times $ less power than CPU, and $31 \times $ less energy than GPU to process one image.

An Optimized GPU Implementation of the MVDR Beamformer for Active Sonar Imaging

The minimum variance distortionless response (MVDR) beamformer has recently been proposed as an attractive alternative to conventional beamformers in active sonar imaging. Unfortunately, it is very computationally complex because a spatial covariance matrix must be estimated and inverted for each image pixel. This may discourage its unnecessary use in sonar systems which are continuously being pushed to ever higher imaging ranges and resolutions. In this study, we show that for active sonar systems up to 32 channels, the computation time can be significantly reduced by performing arithmetic optimizations, and by implementing the MVDR beamformer on a graphics processing unit (GPU). We point out important hardware limitations for these devices, and assess the design in terms of how efficiently it is able to use the GPU's resources. On a quad-core Intel Xeon system with a high-end Nvidia GPU, our GPU implementation renders more than a million pixels per second (1 MP/s). Compared to our initial central processing unit (CPU) implementation, the optimizations described herein led to a speedup of more than two orders of magnitude, or an expected five to ten times improvement had the CPU received similar optimization effort. This throughput enables real-time processing of sonar data, and makes the MVDR a viable alternative to conventional methods in practical systems.

NMF-mGPU: non-negative matrix factorization on multi-GPU systems

BackgroundIn the last few years, the Non-negative Matrix Factorization ( NMF ) technique has gained a great interest among the Bioinformatics community, since it is able to extract interpretable parts from high-dimensional datasets. However, the computing time required to process large data matrices may become impractical, even for a parallel application running on a multiprocessors cluster.In this paper, we present NMF-mGPU, an efficient and easy-to-use implementation of the NMF algorithm that takes advantage of the high computing performance delivered by Graphics-Processing Units ( GPUs ). Driven by the ever-growing demands from the video-games industry, graphics cards usually provided in PCs and laptops have evolved from simple graphics-drawing platforms into high-performance programmable systems that can be used as coprocessors for linear-algebra operations. However, these devices may have a limited amount of on-board memory, which is not considered by other NMF implementations on GPU.Results NMF-mGPU is based on CUDA ( Compute Unified Device Architecture ), the NVIDIA’s framework for GPU computing. On devices with low memory available, large input matrices are blockwise transferred from the system’s main memory to the GPU’s memory, and processed accordingly. In addition, NMF-mGPU has been explicitly optimized for the different CUDA architectures. Finally, platforms with multiple GPUs can be synchronized through MPI ( Message Passing Interface ). In a four-GPU system, this implementation is about 120 times faster than a single conventional processor, and more than four times faster than a single GPU device (i.e., a super-linear speedup).ConclusionsApplications of GPUs in Bioinformatics are getting more and more attention due to their outstanding performance when compared to traditional processors. In addition, their relatively low price represents a highly cost-effective alternative to conventional clusters. In life sciences, this results in an excellent opportunity to facilitate the daily work of bioinformaticians that are trying to extract biological meaning out of hundreds of gigabytes of experimental information. NMF-mGPU can be used “out of the box” by researchers with little or no expertise in GPU programming in a variety of platforms, such as PCs, laptops, or high-end GPU clusters. NMF-mGPU is freely available at https://github.com/bioinfo-cnb/bionmf-gpu.