GPGPU Research Articles

Approximate Cache in GPGPUs

There is a growing number of application domains ranging from multimedia to machine learning where a certain level of inexactness can be tolerated. For these applications, approximate computing is an effective technique that trades off some loss in output data integrity for energy and/or performance gains. In this article, we present the approximate cache, which approximates similar values and saves energy in the L2 cache of general-purpose graphics processing units (GPGPUs). The L2 cache is a critical component in memory hierarchy of GPGPUs, as it accommodates data of thousands of simultaneously executing threads. Simply increasing the size of the L2 cache is not a viable solution to keep up with the growing size of data in many-core applications. This work is motivated by the observation that threads within a warp write values into memory that are arithmetically similar. We exploit this property and propose a low-cost and implementation-efficient hardware to trade off accuracy for energy. The approximate cache identifies similar values during the runtime and allows only one thread writes into the cache in the event of similarity. Since the approximate cache is able to pack more data in a smaller space, it enables downsizing of the data array with negligible impact on cache misses and lower-level memory. The approximate cache reduces both dynamic and static energy. By storing data of a thread into a cache block, each memory instruction requires accessing fewer cache cells, thus reducing dynamic energy. In addition, the approximate cache increases frequency of bank idleness. By power gating idle banks, static energy is reduced. Our evaluations reveal that the approximate cache reduces energy by 52% with minimal quality degradation while maintaining performance of a diverse set of GPGPU applications.

Real time object detection and trackingsystem for video surveillance system

This paper introduces a system capable of real-time video surveillance in low-end edge computing environment by combining object detection tracking algorithm. Recently, the accuracy of object detection has been improved due to the performance of approaches based on deep learning algorithm such as region-based convolutional network, which has two stages for inferencing. One-stage detection algorithms such as single shot detector and you only look once (YOLO) have been developed at the expense of some accuracy and can be used for real-time systems. However, high-performance hardware such as general-purpose graphics processing unit is required to achieve excellent object detection performance and speed. In this study, we propose an approach called N-YOLO which is instead of resizing image step in YOLO algorithm, it divides into fixed size images used in YOLO and merges detection results of each divided sub-image with inference results at different times using correlation-based tracking algorithm the amount of computation for object detection and tracking can be significantly reduced. In addition, we propose a system that can guarantee real-time performance in various edge computing environments by adaptively controlling the cycle of object detection and tracking.

Cooperation of CUDA and Intel multi-core architecture in the independent component analysis algorithm for EEG data

AbstractObjectivesThe electroencephalographic signal is largely exposed to external disturbances. Therefore, an important element of its processing is its thorough cleaning.MethodsOne of the common methods of signal improvement is the independent component analysis (ICA). However, it is a computationally expensive algorithm, hence methods are needed to decrease its execution time. One of the ICA algorithms (fastICA) and parallel computing on the CPU and GPU was used to reduce the algorithm execution time.ResultsThis paper presents the results of study on the implementation of fastICA, which uses some multi-core architecture and the GPU computation capabilities.ConclusionsThe use of such a hybrid approach shortens the execution time of the algorithm.

Optimizing non-coalesced memory access for irregular applications with GPU computing

General purpose graphics processing units (GPGPUs) can be used to improve computing performance considerably for regular applications. However, irregular memory access exists in many applications, and the benefits of graphics processing units (GPUs) are less substantial for irregular applications. In recent years, several studies have presented some solutions to remove static irregular memory access. However, eliminating dynamic irregular memory access with software remains a serious challenge. A pure software solution without hardware extensions or offline profiling is proposed to eliminate dynamic irregular memory access, especially for indirect memory access. Data reordering and index redirection are suggested to reduce the number of memory transactions, thereby improving the performance of GPU kernels. To improve the efficiency of data reordering, an operation to reorder data is offloaded to a GPU to reduce overhead and thus transfer data. Through concurrently executing the compute unified device architecture (CUDA) streams of data reordering and the data processing kernel, the overhead of data reordering can be reduced. After these optimizations, the volume of memory transactions can be reduced by 16.7%–50% compared with CUSPARSE-based benchmarks, and the performance of irregular kernels can be improved by 9.64%–34.9% using an NVIDIA Tesla P4 GPU.

Comprehensive high-speed reconciliation for continuous-variable quantum key distribution

Reconciliation is currently the bottleneck of continuous-variable quantum key distribution systems for its great influence on the key rate and the distance of systems. In this paper, we address the increase in key rates by accelerating the speed of reconciliation algorithms based on the protocol of sliced error correction on a heterogeneous computing structure (a GPGPU card (general purpose graphics processing units will be abbreviated as GPU in this paper) and a general CPU) in the framework of Open Computing Language (OpenCL) (OpenCL is a programming framework based on C language). A block length of its component codes of low-density parity-check (LDPC) codes up to 2 $$^{17}$$ bits is employed in order to achieve a higher reconciliation efficiency. To meet the requirements of the OpenCL specifications, we designed a data structure, namely static cross bi-directional circular linked list, to store a super large sparse check matrix of the LDPC codes. Such a configuration ensures the practicability of our system, i.e. a better trade-off between the speed and net key rates of the reconciliation. The speed of the proposed reconciliation scheme reaches about 70.1 Mb/s with 512 codewords decoding in parallel, approximately 3600 times faster than that with the platform with only a CPU.

Acceleration and enhancement of reliability of simulated annealing for optimizing thinning schedule of a forest stand

Although a simulated-annealing (SA) based method for exploring the optimal thinning schedule in a forest stand has been developed, its long calculation time limits its applicability. Furthermore, the method occasionally fails to provide a sufficient optimal solution even with multiple iterations. Therefore, this study aims to accelerate and enhance the reliability of the aforementioned method. To enhance the reliability, a new neighborhood search method and meta-optimization technique of the parameter sets of the SA were developed. To accelerate the calculation, the technique of general-purpose computing on GPU (GPGPU) was utilized. Using 16 harvesting models involving 20 rotation ages, an effective implementation was identified. Although the ordinary method occasionally failed to provide solutions that exceeded the quality of the solutions provided using a full-search method, the new neighborhood search method provided better solutions with reasonable number of iterations for all cases. The GPGPU allowed up to 28 times acceleration with a GPU, compared with a program that uses only one 28-thread CPU. For a model with 20 candidate rotation ages, the developed method can provide practically sufficient optimal solutions within 6 s for a simple yielding model and within 1 min for a complex logging yielding model.

Passively parallel regularized stokeslets.

Stokes flow, discussed by G.G. Stokes in 1851, describes many microscopic biological flow phenomena, including cilia-driven transport and flagellar motility; the need to quantify and understand these flows has motivated decades of mathematical and computational research. Regularized stokeslet methods, which have been used and refined over the past 20 years, offer significant advantages in simplicity of implementation, with a recent modification based on nearest-neighbour interpolation providing significant improvements in efficiency and accuracy. Moreover this method can be implemented with the majority of the computation taking place through built-in linear algebra, entailing that state-of-the-art hardware and software developments in the latter, in particular multicore and GPU computing, can be exploited through minimal modifications (‘passive parallelism’) to existing Matlab computer code. Hence, and with widely available GPU hardware, significant improvements in the efficiency of the regularized stokeslet method can be obtained. The approach is demonstrated through computational experiments on three model biological flows: undulatory propulsion of multiple Caenorhabditis elegans, simulation of progression and transport by multiple sperm in a geometrically confined region, and left–right symmetry breaking particle transport in the ventral node of the mouse embryo. In general an order-of-magnitude improvement in efficiency is observed. This development further widens the complexity of biological flow systems that are accessible without the need for extensive code development or specialist facilities.This article is part of the theme issue ‘Stokes at 200 (part 2)’.

An Adaptive Clock Scheme Exploiting Instruction-Based Dynamic Timing Slack for a GPGPU Architecture

This article presents an adaptive clock scheme to exploit instruction-based dynamic timing slack (DTS) for a general-purpose graphics processor unit (GPGPU) architecture. Based on the developed transitional static timing analysis, the deterministic DTS can be identified for each instruction at different pipeline stages. A critical path (CP) messenger scheme was designed to monitor the runtime utilization of CPs. Both real-time issued instruction information and CP messengers are utilized to determine the runtime DTS margin and guide the cycle-by-cycle clock period adjustment. To apply the proposed adaptive clock on GPGPU, a hierarchical clocking scheme is built including a global phase-locked loop (PLL) and local delay-locked loop (DLL)-based clock generator inside each compute unit (CU). Each CU core contains its own clock domain with adjustable local clocking. In addition, to exploit error-resilient characteristics of the neural network, an elastic pipeline clocking scheme is developed to redistribute the timing margin across pipeline stages for machine learning computations. Measurement results from the implemented open-source GPGPU architecture on a 65 nm CMOS process demonstrate up to 18% performance improvement or equivalent 30% energy saving can be obtained by exploiting the deterministic instruction-based DTS. The proposed elastic pipeline clocking can gain an additional 8% energy saving with small accuracy degradation for neural network inference operations.

Special issue on SoC and AI processors

Special issue on SoC and AI processors

AB9: A neural processor for inference acceleration

We present AB9, a neural processor for inference acceleration. AB9 consists of a systolic tensor core (STC) neural network accelerator designed to accelerate artificial intelligence applications by exploiting the data reuse and parallelism characteristics inherent in neural networks while providing fast access to large on-chip memory. Complementing the hardware is an intuitive and user-friendly development environment that includes a simulator and an implementation flow that provides a high degree of programmability with a short development time. Along with a 40-TFLOP STC that includes 32k arithmetic units and over 36 MB of on-chip SRAM, our baseline implementation of AB9 consists of a 1-GHz quad-core setup with other various industry-standard peripheral intellectual properties. The acceleration performance and power efficiency were evaluated using YOLOv2, and the results show that AB9 has superior performance and power efficiency to that of a general-purpose graphics processing unit implementation. AB9 has been taped out in the TSMC 28-nm process with a chip size of 17 × 23 mm2. Delivery is expected later this year.

One size does not fit all: accelerating OLAP workloads with GPUs

GPU has been considered as one of the next-generation platforms for real-time query processing databases. In this paper we empirically demonstrate that the representative GPU databases [e.g., OmniSci (Open Source Analytical Database & SQL Engine, https://www.omnisci.com/platform/omniscidb, 2019)] may be slower than the representative in-memory databases [e.g., Hyper (Neumann and Leis, IEEE Data Eng Bull 37(1):3–11, 2014)] with typical OLAP workloads (with Star Schema Benchmark) even if the actual dataset size of each query can completely fit in GPU memory. Therefore, we argue that GPU database designs should not be one-size-fits-all; a general-purpose GPU database engine may not be well-suited for OLAP workloads without careful designed GPU memory assignment and GPU computing locality. In order to achieve better performance for GPU OLAP, we need to re-organize OLAP operators and re-optimize OLAP model. In particular, we propose the 3-layer OLAP model to match the heterogeneous computing platforms. The core idea is to maximize data and computing locality to specified hardware. We design the vector grouping algorithm for data-intensive workload which is proved to be assigned to CPU platform adaptive. We design the TOP-DOWN query plan tree strategy to guarantee the optimal operation in final stage and pushing the respective optimizations to the lower layers to make global optimization gains. With this strategy, we design the 3-stage processing model (OLAP acceleration engine) for hybrid CPU-GPU platform, where the computing-intensive star-join stage is accelerated by GPU, and the data-intensive grouping & aggregation stage is accelerated by CPU. This design maximizes the locality of different workloads and simplifies the GPU acceleration implementation. Our experimental results show that with vector grouping and GPU accelerated star-join implementation, the OLAP acceleration engine runs 1.9×, 3.05× and 3.92× faster than Hyper, OmniSci GPU and OmniSci CPU in SSB evaluation with dataset of SF = 100.

Parallel Algorithms for Solving Linear Systems on Hybrid Computers

Introduction. At present, in science and technology, new computational problems constantly arise with large volumes of data, the solution of which requires the use of powerful supercomputers. Most of these problems come down to solving systems of linear algebraic equations (SLAE). The main problem of solving problems on a computer is to obtain reliable solutions with minimal computing resources. However, the problem that is solved on a computer always contains approximate data regarding the original task (due to errors in the initial data, errors when entering numerical data into the computer, etc.). Thus, the mathematical properties of a computer problem can differ significantly from the properties of the original problem. It is necessary to solve problems taking into account approximate data and analyze computer results. Despite the significant results of research in the field of linear algebra, work in the direction of overcoming the existing problems of computer solving problems with approximate data is further aggravated by the use of contemporary supercomputers, do not lose their significance and require further development. Today, the most high-performance supercomputers are parallel ones with graphic processors. The architectural and technological features of these computers make it possible to significantly increase the efficiency of solving problems of large volumes at relatively low energy costs. The purpose of the article is to develop new parallel algorithms for solving systems of linear algebraic equations with approximate data on supercomputers with graphic processors that implement the automatic adjustment of the algorithms to the effective computer architecture and the mathematical properties of the problem, identified in the computer, as well with estimates of the reliability of the results. Results. A methodology for creating parallel algorithms for supercomputers with graphic processors that implement the study of the mathematical properties of linear systems with approximate data and the algorithms with the analysis of the reliability of the results are described. The results of computational experiments on the SKIT-4 supercomputer are presented. Conclusions. Parallel algorithms have been created for investigating and solving linear systems with approximate data on supercomputers with graphic processors. Numerical experiments with the new algorithms showed a significant acceleration of calculations with a guarantee of the reliability of the results. Keywords: systems of linear algebraic equations, hybrid algorithm, approximate data, reliability of the results, GPU computers.

Soil Monitor: an internet platform to challenge soil sealing in Italy

AbstractThis work proposes a new type of web‐based geospatial decision support system—called Soil Monitor—which provides a multidisciplinary operational tool useful for a multi‐stakeholder community to challenge soil sealing. Various technological and technical features of Soil Monitor are presented and discussed with particular reference to the combination of WebGIS with on‐the‐fly geospatial processing based on GPU computing, specifically designed to allow real‐time requests. In addition, we present three quantitative analysis of soil sealing at three different scales. The study at national level illustrated that the type of land use strongly affected soil‐sealing dynamics. Indeed, the class complex cultivation pattern resulted the most fragile land use while forests resulted the most resilient one. The study at province level illustrated the key importance of having on‐the‐fly the map of rural fragmentation for any Italian province. This map—quantifying the landscape integrity of the rural and natural areas—enables to better locate both new green corridors or new urban developments minimizing the environmental impact of human activities. The study at municipality level demonstrated the importance of quantifying a pool of spatial planning indicators and illustrated that (a) the largest urban dispersion occurs in hilly touristic municipalities close to the sea, (b) the increase in soil sealing in rural sites is not related to population growth. As a result, Soil Monitor is deemed useful both to support decision making at different spatial scales and to raise awareness in people, professionals, and experts of other fields. Keywords soil sealing, geospatial decision support system, compute unified device architecture, graphical processing unit computing, urban planning.

PowerCoord: Power capping coordination for multi-CPU/GPU servers using reinforcement learning

Modern supercomputers and cloud providers rely on server nodes that are equipped with multiple CPU sockets and general purpose GPUs (GPGPUs) to handle the high demand for intensive computations. These nodes consume much higher power than commodity servers, and integrating them with power capping systems used in modern clusters presents new challenges. In this paper, we propose a new power capping controller, PowerCoord, that is specifically designed for servers with multiple CPU and GPU sockets that are running multiple jobs at a time. PowerCoord coordinates among the various power domains (e.g., CPU sockets and GPUs) inside a node server to meet target power caps, while seeking to maximize throughput. Our approach also takes into consideration job deadlines and priorities. Because performance modeling for co-located jobs is error-prone, PowerCoord uses a learning method. PowerCoord has a number of heuristic policies to allocate power among the various CPUs and GPUs, and it uses reinforcement learning for policy selection during runtime. Based on the observed state of the system, PowerCoord shifts the distribution of selected policies. We implement our power cap controller on a real multi-CPU/GPU server with low overhead, and we demonstrate that it is able to meet target power caps while maximizing the throughput, and balancing other demands such as priorities and deadlines. Our results show PowerCoord improves the server throughput on average by 18% compared with the case when power is not coordinated among CPU/GPU domains. Also, PowerCoord improves the server throughput on average by 11% compared with prior work that uses a heuristic approach to coordinate the power among domains.

Comparison of Baseband Processors in Terms of Realization SDR-Transceivers

Software-defined Radio is a programmable transceiver with the capability of operating various wireless communication protocols without the need to change or update the hardware. Consequently, Software-defined Radio has earned a lot of attention and is of great significance to both academia, military and aerospace industry. Components of Software-defined Radio (e.g. mixers, filters, amplifiers, modulators/demodulators, detectors, etc.) implemented by means of software on a personal computer or embedded system. Operation of signal processing are handed over to the baseband processor, rather than being done in special electronic circuits. Baseband processors are implemented through employing various types of hardware platforms, such as General Purpose Processors, Graphics Processing Units, Digital Signal Processors, and Field Programmable Gate Arrays. Each of these platforms is associated with their own set of advantages and disadvantages. In this paper was proposed a comparison of the state-of-the-art hardware platforms in the context of implementation Software-defined Radio transceivers. For comparison was determined as follow criteria: computational power of hardware platform, power consumption, complexity of developing, and cost of tools and equipment. First approaches to realizing baseband processors is using a General Purpose Processor and accelerating by Graphics Processing Units. But General Purpose Processor and Graphics Processing Units execute software instructions in the sequential order. For this reason, General Purpose Processors are not convenient for high-throughput computing with real-time requirements. Also this hardware platforms have increased power consumption. This aspect does not allow use General Purpose Processor and Graphics Processing Units in small and portable Software-defined Radio transceivers. In other hand, General Purpose Processors are preferable hardware platform by researchers and beginners due to their flexibility and programmability. Therefore, General Purpose Processors and Graphics Processing Units is highly recommended for prototyping Software-defined Radio platforms. Digital Signal Processor was reviewed as alternative approach for implementing baseband processors. Digital Signal Processors is a particular type of General Purpose Processors that is optimized to process digital signals. Digital Signal Processors have similar disadvantage with insufficient computational power, but some manufacturer sell energy optimized Digital Signal Processors. Consequently, Digital Signal Processor is commonly used in small and portable Software-defined Radio transceivers. Field Programmable Gate Arrays and System-on-Chips with Field Programmable Gate Array are strongly recommended for high-performance Software-defined Radio platforms. This hardware platforms combine the flexibility of processors and efficiency of small Digital Signal Processor. Field Programmable Gate Arrays can achieve a high level of parallelism in executing digital signal processing. However, the designers must have a high degree in digital electronics and good acknowledgement of hardware description languages. After the research, was proposed own flexible architecture Software-defined Radio transceiver and methods for development.

Computation of minimal unsatisfiable subformulas for SAT-based digital circuit error diagnosis

The explanation of infeasibilities formed in Minimal Unsatisfiable Subformulas (MUSes) is a core task in the analysis of over-constrained Boolean formulas. A wide range of applications necessitate MUS detection including knowledge-based validation, software design, verification and error diagnosis in digital VLSI circuits. Consequently, various enhanced algorithms for determining MUS have been utilized for solving Maximum Satisfiability algorithms and Conjunction Normal Form (CNF) redundancies. Three enhancements are proposed in this paper. The first is a CPU-GPU algorithm for computation of Minimal Correction Subsets (MCSes) optimized for NVIDIA General Purpose Graphics Processing Unit paradigm. The proposed algorithm of generating all MCSes from the encoded CNF instance was developed using our parallel SSGPU solver and implemented using CUDA. The second enhancement is an algorithm for MUS computation based on auto-reduction of the enhanced MCSes for faster MUS detection. The third improved algorithm is for computing MUS directly without using MCSes. The two proposed algorithms outperform techniques as they could locate and explain design faults in digital VLSI circuits in earlier stages of IC design flow. The second proposed routine of MUS extraction was performed by avoiding a non-critical step in calling the SAT solver during MUS computation, leading to improving the performance of the MUS extraction algorithm. The third proposed mechanism for direct extraction of MUS was optimized by reducing the required SAT-solver calls using a classification of clauses in the input formula. Comparative analysis of the proposed algorithm against the Compute All Minimal Unsatisfiable Subsets (CAMUS) algorithm determined 1.54 × faster detection of MUS using ISCAS'85, ISCAS'89 and synthetic benchmarks. Also, the third algorithm for direct MUS computation delivers 17.05 × faster than shrinking algorithm used in MUST (minimal unsatisfiable subset) tool using ISCAS'85 and synthetic benchmarks. Moreover, it was observed that the CPU-GPU algorithm for MCSes computation based on the SSGPU solver delivered 1.92 × faster than its conventional counterpart, based on CUD@SAT equivalent on GPU using ISCAS'85, ISCAS'89 and synthetic Benchmarks.

Exceeding Conservative Limits: A Consolidated Analysis on Modern Hardware Margins

Modern large-scale computing systems (data centers, supercomputers, cloud and edge setups and high-end cyber-physical systems) employ heterogeneous architectures that consist of multicore CPUs, general-purpose many-core GPUs, and programmable FPGAs. The effective utilization of these architectures poses several challenges, among which a primary one is power consumption. Voltage reduction is one of the most efficient methods to reduce power consumption of a chip. With the galloping adoption of hardware accelerators (i.e., GPUs and FPGAs) in large datacenters and other large-scale computing infrastructures, a comprehensive evaluation of the safe voltage reduction levels for each different chip can be employed for efficient reduction of the total power. We present a survey of recent studies in voltage margins reduction at the system level for modern CPUs, GPUs and FPGAs. The pessimistic voltage guardbands inserted by the silicon vendors can be exploited in all devices for significant power savings. On average, voltage reduction can reach 12% in multicore CPUs, 20% in manycore GPUs and 39% in FPGAs.

Applying the swept rule for solving explicit partial differential equations on heterogeneous computing systems

Applications that exploit the architectural details of high-performance computing (HPC) systems have become increasingly invaluable in academia and industry over the past two decades. The most important hardware development of the last decade in HPC has been the general purpose graphics processing unit (GPGPU), a class of massively parallel devices that now contributes the majority of computational power in the top 500 supercomputers. As these systems grow, small costs such as latency—due to the fixed cost of memory accesses and communication—accumulate in a large simulation and become a significant barrier to performance. The swept time-space decomposition rule is a communication-avoiding technique for time-stepping stencil update formulas that attempts to reduce latency costs. This work extends the swept rule by targeting heterogeneous, CPU/GPU architectures representing current and future HPC systems. We compare our approach to a naive decomposition scheme with two test equations using an MPI+CUDA pattern on 40 processes over two nodes containing one GPU. The swept rule produces a factor of 1.9 to 23 speedup for the heat equation and a factor of 1.1 to 2.0 speedup for the Euler equations, using the same processors and work distribution, and with the best possible configurations. These results show the potential effectiveness of the swept rule for different equations and numerical schemes on massively parallel compute systems that incur substantial latency costs.

Simulation of Fire with a Gas Kinetic Scheme on Distributed GPGPU Architectures

The simulation of fire is a challenging task due to its occurrence on multiple space-time scales and the non-linear interaction of multiple physical processes. Current state-of-the-art software such as the Fire Dynamics Simulator (FDS) implements most of the required physics, yet a significant drawback of this implementation is its limited scalability on modern massively parallel hardware. The current paper presents a massively parallel implementation of a Gas Kinetic Scheme (GKS) on General Purpose Graphics Processing Units (GPGPUs) as a potential alternative modeling and simulation approach. The implementation is validated for turbulent natural convection against experimental data. Subsequently, it is validated for two simulations of fire plumes, including a small-scale table top setup and a fire on the scale of a few meters. We show that the present GKS achieves comparable accuracy to the results obtained by FDS. Yet, due to the parallel efficiency on dedicated hardware, our GKS implementation delivers a reduction of wall-clock times of more than an order of magnitude. This paper demonstrates the potential of explicit local schemes in massively parallel environments for the simulation of fire.

Trivial Bypassing in GPGPUs

This letter presents trivial bypassing to detect and skip execution of trivial instructions in general-purpose graphics processing units (GPGPUs). During the execution of a program, a significant number of instructions are trivial; that is, the instructions do not need functional units for execution. The outcome of such instructions can be determined based on source operands. Execution of these instructions unnecessarily wastes hardware resources and reduces energy efficiency of GPGPUs. We propose a microarchitectural technique that detects and bypasses trivial instructions dynamically and during the runtime. By bypassing trivial instructions, the energy of functional units is reduced. In addition to functional units, the other component that benefits from trivial bypassing is the register file. We exploit register renaming to remap destination field of trivial instructions. Using register renaming, logical registers with the same values share the same physical registers. This reduces the number of accesses to physical registers and enhances the energy efficiency of GPGPUs. Evaluations using a wide range of applications reveal that trivial bypassing reduces the energy of GPGPUs by 8% with negligible impact on performance.