GPU Implementation Research Articles

Automated CNN back-propagation pipeline generation for FPGA online training

Training of convolutional neural networks (CNNs) on embedded platforms to support on-device learning has become essential for the future deployment of CNNs on autonomous systems. In this work, we present an automated CNN training pipeline compilation tool for Xilinx FPGAs. We automatically generate multiple hardware designs from high-level CNN descriptions using a multi-objective optimization algorithm that explores the design space by exploiting CNN parallelism. These designs that trade-off resources for throughput allow users to tailor implementations to their hardware and applications. The training pipeline is generated based on the backpropagation (BP) equations of convolution which highlight an overlap in computation. We translate the overlap into hardware by reusing most of the forward pass (FP) pipeline reducing the resources overhead. The implementation uses a streaming interface that lends itself well to data streams and live feeds instead of static data reads from memory. Meaning, we do not use the standard array of processing elements (PEs) approach, which is efficient for offline inference, instead we translate the architecture into a pipeline where data is streamed through allowing for new samples to be read as they become available. We validate the results using the Zynq-7100 on three datasets and varying size architectures against CPU and GPU implementations. GPUs consistently outperform FPGAs in training times in batch processing scenarios, but in data stream scenarios, FPGA designs achieve a significant speedup compared to GPU and CPU when enough resources are dedicated to the learning task. A 2.8×, 5.8×, and 3× speed up over GPU was achieved on three architectures trained on MNIST, SVHN, and CIFAR-10 respectively.

Signal processing for a reverse‐GPS wildlife tracking system: CPU and GPU implementation experiences

AbstractWe present robust high‐performance implementations of signal‐processing tasks performed by a high‐throughput wildlife tracking system called ATLAS. The system tracks radio transmitters attached to wild animals by estimating the time of arrival of radio packets to multiple receivers (base stations). Time‐of‐arrival estimation of wideband radio signals is computationally expensive, especially in acquisition mode (when the time of transmission is not known, not even approximately). These computations are a bottleneck that limits the throughput of the system. We developed a sequential high‐performance CPU implementation of the computations a few years back, and more recently a GPU implementation. Both strive to balance performance with simplicity, maintainability, and development effort, as most real‐world codes do. The article reports on the two implementations and carefully evaluates their performance. The evaluations indicates that the GPU implementation dramatically improves performance and power‐performance relative to the sequential CPU implementation running on a desktop CPU typical of the computers in current base stations. Performance improves by more than 50X on a high‐end GPU and more than 4X with a GPU platform that consumes almost 5 times less power than the CPU platform. Performance‐per‐Watt ratios also improve (by more than 16X), and so do the price‐performance ratios.

A direct Jacobian total Lagrangian explicit dynamics finite element algorithm for real‐time simulation of hyperelastic materials

AbstractThis article presents a novel direct Jacobian total Lagrangian explicit dynamics (DJ‐TLED) finite element algorithm for real‐time nonlinear mechanics simulation. The nodal force contributions are expressed using only the Jacobian operator, instead of the deformation gradient tensor and finite deformation tensor, for fewer computational operations at run‐time. Owing to this proposed Jacobian formulation, novel expressions are developed for strain invariants and constant components, which are also based on the Jacobian operator. Results show that the proposed DJ‐TLED consumed between 0.70× and 0.88× CPU solution times compared to state‐of‐the‐art TLED and achieved up to 121.72× and 94.26× speed improvements in tetrahedral and hexahedral meshes, respectively, using GPU acceleration. Compared to TLED, the most notable difference is that the notions of stress and strain are not explicitly visible in the proposed DJ‐TLED but embedded implicitly in the formulation of nodal forces. Such a force formulation can be beneficial for fast deformation computation and can be particularly useful if the displacement field is of primary interest, which is demonstrated using a neurosurgical simulation of brain deformations for image‐guided neurosurgery. The present work contributes towards a comprehensive DJ‐TLED algorithm concerning isotropic and anisotropic hyperelastic constitutive models and GPU implementation.

Laplacian pyramid based non-linear coherence diffusion for real-time ultrasound image speckle reduction

The inherent speckle noise in medical ultrasound image degrades valuable clinical information and affects the diagnosis quality. Therefore, commercial ultrasound systems always face a big challenge to improve the visual quality of ultrasound images by reducing the speckle noise. A new hybrid algorithm is proposed in the present paper for speckle reduction by combining nonlinear coherence diffusion (NCD) with pyramid-transform-based multi-scale filters (named as Laplacian Pyramid Based Non-linear Coherence Diffusion; viz. LPNCD). The new approach exhibits good capability in preserving tissue features while enhancing the coherence. The theory and implementation of the approach are presented and verified based on phantom and clinical data. By comparing with previous techniques on quantified signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR), the results suggest that the LPNCD method may demonstrate superior performance than other commonly used AD-class filters. Meanwhile, the performance of GPU implementations of the current technique is also evaluated, and it is verified that the LPNCD approach may have good performance efficiency and wonderful visual effect as running on mainstream commercial ultrasound systems.

Fast linking numbers for topology verification of loopy structures

It is increasingly common to model, simulate, and process complex materials based on loopy structures, such as in yarn-level cloth garments, which possess topological constraints between inter-looping curves. While the input model may satisfy specific topological linkages between pairs of closed loops, subsequent processing may violate those topological conditions. In this paper, we explore a family of methods for efficiently computing and verifying linking numbers between closed curves, and apply these to applications in geometry processing, animation, and simulation, so as to verify that topological invariants are preserved during and after processing of the input models. Our method has three stages: (1) we identify potentially interacting loop-loop pairs, then (2) carefully discretize each loop's spline curves into line segments so as to enable (3) efficient linking number evaluation using accelerated kernels based on either counting projected segment-segment crossings, or by evaluating the Gauss linking integral using direct or fast summation methods (Barnes-Hut or fast multipole methods). We evaluate CPU and GPU implementations of these methods on a suite of test problems, including yarn-level cloth and chainmail, that involve significant processing: physics-based relaxation and animation, user-modeled deformations, curve compression and reparameterization. We show that topology errors can be efficiently identified to enable more robust processing of loopy structures.

Fast Low-Precision Computer-Generated Holography on GPU

Computer-generated holography (CGH) is a notoriously difficult computation problem, simulating numerical diffraction, where every scene point can affect every hologram pixel. To tackle this challenge, specialized software instructions and hardware solutions are developed to significantly reduce calculation time and power consumption. In this work, we propose a novel algorithm for high-performance point-based CGH, leveraging fixed-point integer representations, the separability of the Fresnel transform and using new look-up table free cosine representation. We report up to a 3-fold speed up over an optimized floating-point GPU implementation, as well as a 15 dB increase in quality over a state-of-the-art FPGA-based fixed-point integer solution.

Architecture-Aware Modeling of Pedestrian Dynamics.

The spread of infectious diseases arises from complex interactions between disease dynamics and human behavior. Predicting the outcome of this complex system is difficult. Consequently, there has been a recent emphasis on comparing the relative risks of different policy options rather than precise predictions. Here, one performs a parameter sweep to generate a large number of possible scenarios for human behavior under different policy options and identifies the relative risks of different decisions regarding policy or design choices. In particular, this approach has been used to identify effective approaches to social distancing in crowded locations, with pedestrian dynamics used to simulate the movement of individuals. This incurs a large computational load, though. The traditional approach of optimizing the implementation of existing mathematical models on parallel systems leads to a moderate improvement in computational performance. In contrast, we show that when dealing with human behavior, we can create a model from scratch that takes computer architectural features into account, yielding much higher performance without requiring complicated parallelization efforts. Our solution is based on two key observations. (i) Models do not capture human behavior as precisely as models for scientific phenomena describe natural processes. Consequently, there is some leeway in designing a model to suit the computational architecture. (ii) The result of a parameter sweep, rather than a single simulation, is the semantically meaningful result. Our model leverages these features to perform efficiently on CPUs and GPUs. We obtain a speedup factor of around 60 using this new model on two Xeon Platinum 8280 CPUs and a factor 125 speedup on 4 NVIDIA Quadro RTX 5000 GPUs over a parallel implementation of the existing model. The careful design of a GPU implementation makes it fast enough for real-time decision-making. We illustrate it on an application to COVID-19.

Microsimulation analysis for network traffic assignment (MANTA) at metropolitan-scale for agile transportation planning

Abrupt changes in the environment have triggered massive and precipitous changes in human mobility. This requires modeling entire metropolitan areas to recognize the broader effects on the network. However, there is a trade-off between increasing the level of detail of a model and decreasing computational performance. Current implementations compromise by simulating small spatial scales, and those that operate at larger scales often require access to expensive high performance computing systems or have computation times on the order of days or weeks that discourage productive research and planning. This paper introduces a new platform, MANTA (Microsimulation Analysis for Network Traffic Assignment), for traffic microsimulation at the metropolitan-scale, employing a highly efficient and parallelized GPU implementation. The runtime to simulate all morning trips, using half-second timesteps, for the nine-county San Francisco Bay Area is just over four minutes, significantly improving the state of the art in large-scale traffic microsimulation.

ScAMACE: model-based approach to the joint analysis of single-cell data on chromatin accessibility, gene expression and methylation.

The advancement in technologies and the growth of available single-cell datasets motivate integrative analysis of multiple single-cell genomic datasets. Integrative analysis of multimodal single-cell datasets combines complementary information offered by single-omic datasets and can offer deeper insights on complex biological process. Clustering methods that identify the unknown cell types are among the first few steps in the analysis of single-cell datasets, and they are important for downstream analysis built upon the identified cell types. We propose scAMACE for the integrative analysis and clustering of single-cell data on chromatin accessibility, gene expression and methylation. We demonstrate that cell types are better identified and characterized through analyzing the three data types jointly. We develop an efficient Expectation-Maximization algorithm to perform statistical inference, and evaluate our methods on both simulation study and real data applications. We also provide the GPU implementation of scAMACE, making it scalable to large datasets. The software and datasets are available at https://github.com/cuhklinlab/scAMACE_py (python implementation) and https://github.com/cuhklinlab/scAMACE (R implementation). Supplementary data are available at Bioinformatics online.

Research on range alignment in real-time ISAR imaging and GPU implementation

Research on range alignment in real-time ISAR imaging and GPU implementation

Velocity Skinning for Real‐time Stylized Skeletal Animation

AbstractSecondary animation effects are essential for liveliness. We propose a simple, real‐time solution for adding them on top of standard skinning, enabling artist‐driven stylization of skeletal motion. Our method takes a standard skeleton animation as input, along with a skin mesh and rig weights. It then derives per‐vertex deformations from the different linear and angular velocities along the skeletal hierarchy. We highlight two specific applications of this general framework, namely the cartoon‐like “squashy” and “floppy” effects, achieved from specific combinations of velocity terms. As our results show, combining these effects enables to mimic, enhance and stylize physical‐looking behaviours within a standard animation pipeline, for arbitrary skinned characters. Interactive on CPU, our method allows for GPU implementation, yielding real‐time performances even on large meshes. Animator control is supported through a simple interface toolkit, enabling to refine the desired type and magnitude of deformation at relevant vertices by simply painting weights. The resulting rigged character automatically responds to new skeletal animation, without further input.

Restricted Power Diagrams on the GPU

AbstractWe propose a method to simultaneously decompose a 3D object into power diagram cells and to integrate given functions in each of the obtained simple regions. We offer a novel, highly parallel algorithm that lends itself to an efficient GPU implementation. It is optimized for algorithms that need to compute many decompositions, for instance, centroidal Voronoi tesselation algorithms and incompressible fluid dynamics simulations.We propose an efficient solution that directly evaluates the integrals over every cell without computing the power diagram explicitly and without intersecting it with a tetrahedralization of the domain. Most computations are performed on the fly, without storing the power diagram. We manipulate a triangulation of the boundary of the domain (instead of tetrahedralizing the domain) to speed up the process. Moreover, the cells are treated independently one from another, making it possible to trivially scale up on a parallel architecture.Despite recent Voronoi diagram generation methods optimized for the GPU, computing integrals over restricted power diagrams still poses significant challenges; the restriction to a complex simulation domain is difficult and likely to be slow. It is not trivial to determine when a cell of a power diagram is completely computed, and the resulting integrals (e.g. the weighted Laplacian operator matrix) do not fit into fast (shared) GPU memory. We address all these issues and boost the performance of the state‐of‐the‐art algorithms by a factor 2 to 3 for (unrestricted) Voronoi diagrams and a ×50 speed‐up with respect to CPU implementations for restricted power diagrams. An essential ingredient to achieve this is our new scheduling strategy that allows us to treat each Voronoi/power diagram cell with optimal settings and to benefit from the fast memory.

An efficient and locality-oriented Hausdorff distance algorithm: Proposal and analysis of paradigms and implementations

Hausdorff distance (HD) is a popular similarity metric used in the comparison of images or 3D volumes. Although popular, its main weakness is computing power consumption, being one of the slowest set distances. In this work, a novel, parallel and locality-oriented Hausdorff distance implementation is proposed. Novel as it is the first time in the literature that an actual algorithmic implementation using morphological dilations is proposed and thoroughly evaluated. Parallel, as it is more robust in terms of parallelization than the state-of-the-art algorithm and local as it has an intrinsic sensitivity to voxels that are closer in space. This proposal can be faster than the state-of-the-art in several practical cases such as in medical imaging registrations (up to 8 times faster on average in one of the CPU experiments) and is faster in the worst-case (up to 22337 times faster in one of the CPU experiments). Worst-case scenarios and high resolution volumes also favor the proposed approach. Throughout the work, several sequential and parallel CPU and GPU implementations are evaluated and compared.

An Energy-Efficient Accelerator for Rain Removal Based on Convolutional Neural Network

Rain removal is an important operation in intelligent visual surveillance systems. Many rain removal algorithms based on convolution neural networks (CNNs) are rarely deployed on resource constrained devices. One of the limiting factors is that memory access leads to high energy consumption. To reduce memory access during computation, previous works usually use a fixed computation pattern for different layers in CNN. For different and massive input and output feature maps, fixed computation pattern would lower the power efficiency. Thus, we propose a reconfigurable architecture to support different convolution mapping method. We use hybrid data reuse pattern to reduce energy consumption by 2.4-5.9 times over fixed computation pattern. The hardware is synthesized in the area of 9.83 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> at TSMC 65 nm technology and can restore a 1280 ×720 real world image in 0.57 s which achieves 56.1 × and 2.1 × speed-up comparing to CPU and GPU implementations. The simulation results show that the power efficiency is 287.7 GOPS/W running rain removal network.

Accelerating Robot Dynamics Gradients on a CPU, GPU, and FPGA

Computing the gradient of rigid body dynamics is a central operation in many state-of-the-art planning and control algorithms in robotics. Parallel computing platforms such as GPUs and FPGAs can offer performance gains for algorithms with hardware-compatible computational structures. In this letter, we detail the designs of three faster than state-of-the-art implementations of the gradient of rigid body dynamics on a CPU, GPU, and FPGA. Our optimized FPGA and GPU implementations provide as much as a 3.0x end-to-end speedup over our optimized CPU implementation by refactoring the algorithm to exploit its computational features, e.g., parallelism at different granularities. We also find that the relative performance across hardware platforms depends on the number of parallel gradient evaluations required.

DrPlace: A Deep Learning Based Routability-Driven VLSI Placement Algorithm

<p indent=0mm>In order to improve the routability of deep learning based placement algorithms in the VLSI physical design stage, on the basis of open source placement algorithm DREAMPlace, a deep learning based routability-driven VLSI placement algorithm named DrPlace is proposed. The framework of the algorithm model is composed of deep learning based routability-driven global placement, routability-driven legalization, and routability-driven detailed placement. The algorithm adds the pin density function into the global placement model and proposes an efficient GPU implementation of pin density key kernel. According to the experiment results on ISPD 2011 and DAC 2012 benchmark suites, the algorithm improves the routability of the DREAMPlace and is more effective than traditional routability-driven placement algorithms.

FEECA: Design Space Exploration for Low-Latency and Energy-Efficient Capsule Network Accelerators

In the past few years, Capsule Networks (CapsNets) have taken the spotlight compared to traditional convolutional neural networks (CNNs) for image classification. Unlike CNNs, CapsNets have the ability to learn the spatial relationship between features of the images. However, their complexity grows because of their heterogeneous capsule structure and the dynamic routing , which is an iterative algorithm to dynamically learn the coupling coefficients of two consecutive capsule layers. This necessitates specialized hardware accelerators for CapsNets. Moreover, a high-performance and energy-efficient design of CapsNet accelerators requires exploration of different design decisions (such as the size and configuration of the processing array and the structure of the processing elements). Toward this, we make the following key contributions: 1) FEECA , a novel methodology to explore the design space of the (micro)architectural parameters of a CapsNet hardware accelerator and 2) CapsAcc , the first specialized RTL-level hardware architecture to perform CapsNets inference with high performance and high energy efficiency. Our CapsAcc achieves significant performance improvement, compared to an optimized GPU implementation, due to its efficient implementation of key activation functions, such as squash and softmax , and an efficient data reuse for the dynamic routing. The FEECA methodology employs the Non-dominated Sorting Genetic Algorithm (NSGA-II) to explore the Pareto-optimal points with respect to area, performance, and energy consumption. This requires analytical modeling of the number of clock cycles required to perform each operation of the CapsNet inference and the memory accesses to enable a fast yet accurate design space exploration. We synthesized the complete accelerator architecture in a 45-nm CMOS technology using Synopsys design tools and evaluated it for the MNIST benchmark (as done by the original CapsNet paper from Google Brain’s team) and for a more complex data set, the German Traffic Sign Recognition Benchmark (GTSRB).

Implementation and Performance Evaluation of RNS Variants of the BFV Homomorphic Encryption Scheme

Homomorphic encryption is an emerging form of encryption that provides the ability to compute on encrypted data without ever decrypting them. Potential applications include aggregating sensitive encrypted data on a cloud environment and computing on the data in the cloud without compromising data privacy. There have been several recent advances resulting in new homomorphic encryption schemes and optimized variants. We implement and evaluate the performance of two optimized variants, namely Bajard-Eynard-Hasan-Zucca (BEHZ) and Halevi-Polyakov-Shoup (HPS), of the most promising homomorphic encryption scheme in CPU and GPU. The most interesting (and also unexpected) result of our performance evaluation is that the HPS variant in practice scales significantly better (typically by 15-30 percent) with increase in multiplicative depth of the computation circuit than BEHZ, implying that the HPS variant will always outperform BEHZ for most practical applications. For the multiplicative depth of 98, our fastest GPU implementation performs homomorphic multiplication in 51 ms for 128-bit security settings, which is faster by two orders of magnitude than prior results and already practical for cloud environments supporting GPU computations. Large multiplicative depths supported by our implementations are required for applications involving deep neural networks, logistic regression learning, and other important machine learning problems.

GPU-friendly data structures for real time simulation

Simulators for virtual surgery training need to perform complex calculations very quickly to provide realistic haptic and visual interactions with a user. The complexity is further increased by the addition of cuts to virtual organs, such as would be needed for performing tumor resection. A common method for achieving large performance improvements is to make use of the graphics hardware (GPU) available on most general-use computers. Programming GPUs requires data structures that are more rigid than on conventional processors (CPU), making that data more difficult to update. We propose a new method for structuring graph data, which is commonly used for physically based simulation of soft tissue during surgery, and deformable objects in general. Our method aligns all nodes of the graph in memory, independently from the number of edges they contain, allowing for local modifications that do not affect the rest of the structure. Our method also groups memory transfers so as to avoid updating the entire graph every time a small cut is introduced in a simulated organ. We implemented our data structure as part of a simulator based on a meshless method. Our tests show that the new GPU implementation, making use of the new graph structure, achieves a 10 times improvement in computation times compared to the previous CPU implementation. The grouping of data transfers into batches allows for a 80–90% reduction in the amount of data transferred for each graph update, but accounts only for a small improvement in performance. The data structure itself is simple to implement and allows simulating increasingly complex models that can be cut at interactive rates.

Fast Ray Tracing of Scale‐Invariant Integral Surfaces

AbstractScale‐invariant integral surfaces, which are implicit representations of surfaces, provide a way to define smooth surfaces from skeletons with prescribed radii defined at their vertices. We introduce a new rendering pipeline allowing to visualize such surfaces in real‐time. We rely on the distance to skeleton to define a sampling strategy along the camera rays, dividing each ray into sub‐intervals. The proposed strategy is chosen to capture main field variations. Resulting intervals are processed iteratively, relying on two main ingredients; quadratic interpolation and field mapping, to an approximate squared homothetic distance. The first provides efficient root finding while the second increases the precision of the interpolation, and the combination of both results in an efficient processing routine. Finally, we present a GPU implementation that relies on a dynamic data‐structure in order to efficiently generate the intervals along the ray. This data‐structure also serves as an acceleration structure that allows constant time access to the primitives of interest during the processing of a given ray.