K40 GPU Research Articles

Accelerating DNN-based 3D point cloud processing for mobile computing

3D point cloud data, which are produced by various 3D sensors such as LIDAR and stereo cameras, have been widely deployed by industry leaders such as Google, Uber, Tesla, and Mobileye, for mobile robotic applications such as autonomous driving and humanoid robots. Point cloud data, which are composed of reliable depth information, can provide accurate location and shape characteristics for scene understanding, such as object recognition and semantic segmentation. However, deep neural networks (DNNs), which directly consume point cloud data, are particularly computation-intensive because they have to not only perform multiplication-and-accumulation (MAC) operations but also search neighbors from the irregular 3D point cloud data. Such a task goes beyond the capabilities of general-purpose processors in real-time to figure out the solution as the scales of both point cloud data and DNNs increase from application to application. We present the first accelerator architecture that dynamically configures the hardware on-the-fly to match the computation of both neighbor point search and MAC computation for point-based DNNs. To facilitate the process of neighbor point search and reduce the computation costs, a grid-based algorithm is introduced to search neighbor points from a local region of grids. Evaluation results based on the scene recognition and segmentation tasks show that the proposed design harvests 16.4$\times$ higher performance and saves 99.95% of energy than an NVIDIA Tesla K40 GPU baseline in point cloud scene understanding applications.

A Novel Deep Learning Architecture for Deformable Imaging Registration applied to Image-Guided Radiotherapy

Most clinical image-guided radiotherapy workflows make use of alignment algorithms to map a plan to a daily cone-beam CT (CBCT) onto a patient’s simulation CT (simCT). Current methodologies of rigid and deformable registration have not demonstrated accuracy high enough to enable automated alignment or adaptive recontouring. Here we investigate a novel deep-learning framework for diffeomorphic deformable image registration (DIR) and analyze its performance with regard to adaptive recontouring. Our institutional treatment planning system and CBCT directory were queried for head and neck plans that were delivered to patients between 2016-2018. A self-normalizing, generative convolutional deep network was developed using the PyTorch API and trained on a platform with 8 NVIDIA K80 GPUs. Network input consisted of a concatenated simCT/CBCT pair and an output of a deformable vector field (DVF). This DVF was then applied to the simCT to warp the simCT onto the CBCT. The network was trained in an unsupervised fashion on the combination of three losses: the Kullback-Leibler loss to promote spatially smooth transforms, a novel bottleneck feature-map loss to prioritize complex anatomic features, and a pixelwise intensity difference loss to drive spatially accurate image reconstruction. In order to test the performance of the network, patient-specific contours were warped onto the CBCT using the DVF output by the network. Patients who underwent repeat planning during their radiotherapy were sampled and the 3D-dice coefficient of overlap was calculated between their initial contours and replanned contours and between the warped contours and replanned contours. All initial and replanned contours were generated by a board-certified radiation oncologist. A dataset of 48 patients was compiled, yielding 1728 simCT/CBCT pairs and 64 unique radiotherapy plans and structure sets. 36 patients were used for the training set, representing 1246 simCT/CBCT pairs. Training was performed over 100 epochs. Resulting warped scans showed strong registration by visual review and 3D dice comparison of the external contours. Quantitatively, the model showed a significantly improved 3D dice overlap of the warped contours to the replanned ground-truth contours (see table). The inference time on the trained model was less than 4 seconds. Our implementation of a novel deep-learning architecture shows fast, robust DIR in the setting of IGRT. Our results highlight the applicability of the recent advancements in deep-learning to essential components of radiation treatment: patient position verification and creation of treatment volumes. Further validation is required prior to implementing such a workflow in the radiation clinic for rapid IGRT.Abstract 1180; Table 1Initial-to-Replanned Contour 3D-Dice OverlapWarped-to-Replanned Contour 3D-Dice OverlappGTV0.781 ± 0.0620.873 ± 0.0450.041CTV0.822 ± 0.0510.901 ± 0.0330.032External0.907 ± 0.0690.948 ± 0.0510.028 Open table in a new tab

Audio Tagging System using Deep Learning Model

Deep learning has been getting more attention towards the researchers for transforming input data into an effective representation through various learning algorithms. Hence it requires a large and variety of datasets to ensure good performance and generalization. But manually labeling a dataset is really a time consuming and expensive process, limiting its size. Some of websites like YouTube and Freesound etc. provide large volume of audio data along with their metadata. General purpose audio tagging is one of the newly proposed tasks in DCASE that can give valuable insights into classification of various acoustic sound events. The proposed work analyzes a large scale imbalanced audio data for a audio tagging system. The baseline of the proposed audio tagging system is based on Convolutional Neural Network with Mel Frequency Cepstral Coefficients. Audio tagging system is developed with Google Colaboratory on free Telsa K80 GPU using keras, Tensorflow, and PyTorch. The experimental result shows the performance of proposed audio tagging system with an average mean precision of 0.92 .

Fast search of third-order epistatic interactions on CPU and GPU clusters

Genome-Wide Association Studies (GWASs), analyses that try to find a link between a given phenotype (such as a disease) and genetic markers, have been growing in popularity in the recent years. Relations between phenotypes and genotypes are not easy to identify, as most of the phenotypes are a product of the interaction between multiple genes, a phenomenon known as epistasis. Many authors have resorted to different approaches and hardware architectures in order to mitigate the exponential time complexity of the problem. However, these studies make some compromises in order to keep a reasonable execution time, such as limiting the number of genetic markers involved in the interaction, or discarding some of these markers in an initial filtering stage. This work presents MPI3SNP, a tool that implements a three-way exhaustive search for cluster architectures with the aim of mitigating the exponential growth of the run-time. Modern cluster solutions usually incorporate GPUs. Thus, MPI3SNP includes implementations for both multi-CPU and multi-GPU clusters. To contextualize the performance achieved, MPI3SNP is able to analyze an input of 6300 genetic markers and 3200 samples in less than 6 min using 768 CPU cores or 4 min using 8 NVIDIA K80 GPUs. The source code is available at https://github.com/chponte/mpi3snp .

Compressible lattice Boltzmann simulations on high‐performance and low‐cost GeForce GPU

SummaryThe feasibility of compressible lattice Boltzmann simulations on high‐performance and low‐cost NVIDIA GeForce GPU is studied to enhance computational efficiency of compressible lattice Boltzmann by using much cheaper computing hardware. The NVIDIA GeForce GTX 1080 GPU, which has similar single precision operation computing performance with NVIDIA scientific computational Tesla K80 GPU and much lower price (seven to ten times) than the latter, is used in this work. The problem of GeForce GPUs is with low double precision operation performance and lack of error‐correcting code memory, which will influence the accuracy of compressible lattice Boltzmann simulations. To evaluate its feasibility, the double‐distribution‐function lattice Boltzmann method is adopted for compressible fluid flows. Three typical compressible problems are tested, ie, the Riemann problem, the regular shock reflection problem, and the supersonic boundary layer problem. The results by GeForce GPU with single precision operation have numerical fluctuations in the simulations with shock waves. Moreover, GeForce GPUs have similar numerical accuracy with CPUs in the regular shock reflection problem and supersonic boundary layer problem.

Optimized large-message broadcast for deep learning workloads: MPI, MPI+NCCL, or NCCL2?

Traditionally, MPI runtimes have been designed for clusters with a large number of nodes. However, with the advent of MPI+CUDA applications and GPU clusters with a relatively smaller number of nodes, efficient communication schemes need to be designed for such systems. This coupled with new application workloads brought forward by Deep Learning (DL) frameworks like Caffe and Microsoft Cognitive Toolkit (CNTK) pose additional design constraints due to very large message communication of GPU buffers during the training phase. In this context, special-purpose libraries like NVIDIA NCCL have emerged to deal with DL workloads. In this paper, we address these new challenges for MPI runtimes and propose two new designs to deal with them: (1) A pipelined chain (PC) design for MPI_Bcast that provides efficient intra- and inter-node communication of GPU buffers, and (2) A Topology-Aware pipelined chain (TA-PC) design for systems with multiple GPUs to fully exploit all the available PCIe links available within a multi-GPU node.To highlight the benefits of our designs, we present an in-depth performance landscape for the proposed MPI_Bcast (MPI) designs, our earlier NCCL-based MPI_Bcast (MPI+NCCL1) design, and ncclBroadcast (NCCL2) design. The proposed designs offer up to 14 × and 16.6 × better performance than MPI+NCCL1 based solutions for intra- and inter-node broadcast latency, respectively. With the recent introduction of NCCL2 (inter-node capable) library, we have enhanced our performance results by adding comparisons for the proposed MPI_Bcast designs as well as ncclBroadcast (NCCL2) design. We report up to 10 × better performance for small and medium message sizes and comparable performance for large message sizes. We also observed that the TA-PC design is up to 50% better than the PC design for MPI_Bcast to 64 GPUs. Furthermore, we provide application level performance comparison using a CUDA-Aware version of CNTK called CA-CNTK. The proposed MPI_Bcast designs provide up to 7% improvement over MPI+NCCL based solutions for data parallel training of the VGG network on 128 GPUs. We present our performance evaluation on three GPU clusters with diverse characteristics: (1) KESCH; a dense multi-GPU system with 8 K80 GPU cards per node, (2) RI2; with a single K80 GPU card per node, and (3) Owens; with a single P100 GPU per node.

Efficient Toxicity Prediction via Simple Features Using Shallow Neural Networks and Decision Trees

Toxicity prediction of chemical compounds is a grand challenge. Lately, it achieved significant progress in accuracy but using a huge set of features, implementing a complex blackbox technique such as a deep neural network, and exploiting enormous computational resources. In this paper, we strongly argue for the models and methods that are simple in machine learning characteristics, efficient in computing resource usage, and powerful to achieve very high accuracy levels. To demonstrate this, we develop a single task-based chemical toxicity prediction framework using only 2D features that are less compute intensive. We effectively use a decision tree to obtain an optimum number of features from a collection of thousands of them. We use a shallow neural network and jointly optimize it with decision tree taking both network parameters and input features into account. Our model needs only a minute on a single CPU for its training while existing methods using deep neural networks need about 10 min on NVidia Tesla K40 GPU. However, we obtain similar or better performance on several toxicity benchmark tasks. We also develop a cumulative feature ranking method which enables us to identify features that can help chemists perform prescreening of toxic compounds effectively.

GPU-warp based finite element matrices generation and assembly using coloring method

Abstract Finite element method has been successfully implemented on the graphics processing units to achieve a significant reduction in simulation time. In this paper, new strategies for the finite element matrix generation including numerical integration and assembly are proposed by using a warp per element for a given mesh. These strategies are developed using the well-known coloring method. The proposed strategies use a specialized algorithm to realize fine-grain parallelism and efficient use of on-chip memory resources. The warp shuffle feature of Compute Unified Device Architecture (CUDA) is used to accelerate numerical integration. The evaluation of elemental stiffness matrix is further optimized by adopting a partial parallel implementation of numerical integration. Performance evaluations of the proposed strategies are done for three-dimensional elasticity problem using the 8-noded hexahedral elements with three degrees of freedom per node. We obtain a speedup of up to 8.2× over the coloring based assembly by element strategy (using a single thread per element) on NVIDIA Tesla K40 GPU. Also, the proposed strategies achieve better arithmetic throughput and bandwidth. Highlights CUDA Warp based strategies for FE matrix generation and assembly. Performed using coloring method and on linear hexahedral element meshing in 3D. Obtained speedup of 5.17×− 8.2× over single thread per element strategy on GPU. Strategies showed better arithmetic throughput and bandwidth through code profiling.

Performance of a Second Order Electrostatic Particle-in-Cell Algorithm on Modern Many-Core Architectures

In this paper we present the outline of a novel electrostatic, second order Particle-in-Cell (PIC) algorithm, that makes use of 'ghost particles' located around true particle positions in order to represent a charge distribution. We implement our algorithm within EMPIRE-PIC, a PIC code developed at Sandia National Laboratories. We test the performance of our algorithm on a variety of many-core architectures including NVIDIA GPUs, conventional CPUs, and Intel's Knights Landing. Our preliminary results show the viability of second order methods for PIC applications on these architectures when compared to previous generations of many-core hardware. Specifically, we see an order of magnitude improvement in performance for second order methods between the Tesla K20 and Tesla P100 GPU devices, despite only a 4× improvement in the theoretical peak performance between the devices. Although these initial results show a large increase in runtime over first order methods, we hope to be able to show improved scaling behaviour and increased simulation accuracy in the future.

S-Net: a scalable convolutional neural network for JPEG compression artifact reduction

Recent studies have used deep residual convolutional neural networks (CNNs) for JPEG compression artifact reduction. This study proposes a scalable CNN called S-Net. Our approach effectively adjusts the network scale dynamically in a multitask system for real-time operation with little performance loss. It offers a simple and direct technique to evaluate the performance gains obtained with increasing network depth, and it is helpful for removing redundant network layers to maximize the network efficiency. We implement our architecture using the Keras framework with the TensorFlow backend on an NVIDIA K80 GPU server. We train our models on the DIV2K dataset and evaluate their performance on public benchmark datasets. To validate the generality and universality of the proposed method, we created and utilized a new dataset, called WIN143, for over-processed images evaluation. Experimental results indicate that our proposed approach outperforms other CNN-based methods and achieves state-of-the-art performance.

Student cluster competition 2017, team NTHU: Reproducing vectorization of the tersoff multi-body potential on the Intel Skylake and Nvidia P100 architecture

Abstract Markus Hohnerbach et al. recently published a work to optimize the performance of Tersoff potential, which is a computing scheme used in the LAMMPS molecular dynamics (MD) code. The optimization solver was implemented with three different computation precisions, namely single, mixed, and double. As a special activity of the Student Cluster Competition at SC17 conference, we aimed to reproduce their experimental studies of the optimization solvers in terms of accuracy, performance and scalability. We conducted our experiments on a cluster with Intel Xeon Gold 6130 CPUs and Nvidia Tesla P100 GPUs, while the original work was on a cluster with Intel Xeon E5-2650 CPUs and K40 GPUs. Desite of the differences in computing systems, we demonstrate the claims from the original work can be successfully reproduced by showing the solvers with less precision implementation can still achieve high accuracy results, and exhibit good performance speedup and scalability.

Performance and Power Analysis of HPC Workloads on Heterogeneous Multi-Node Clusters

Performance analysis tools allow application developers to identify and characterize the inefficiencies that cause performance degradation in their codes, allowing for application optimizations. Due to the increasing interest in the High Performance Computing (HPC) community towards energy-efficiency issues, it is of paramount importance to be able to correlate performance and power figures within the same profiling and analysis tools. For this reason, we present a performance and energy-efficiency study aimed at demonstrating how a single tool can be used to collect most of the relevant metrics. In particular, we show how the same analysis techniques can be applicable on different architectures, analyzing the same HPC application on a high-end and a low-power cluster. The former cluster embeds Intel Haswell CPUs and NVIDIA K80 GPUs, while the latter is made up of NVIDIA Jetson TX1 boards, each hosting an Arm Cortex-A57 CPU and an NVIDIA Tegra X1 Maxwell GPU.

Accelerating subset sum and lattice based public-key cryptosystems with multi-core CPUs and GPUs

Post-quantum cryptosystems based on subset sum and lattice problems have gained much attention from researchers due to their simple construction, their resistance to quantum attacks, the new potential applications they provide, and above all, the mathematical security proofs that rigorously relate them to computational hard problems. However, the computational complexity of these cryptosystems is still high compared to classic number-theoretical ones, which may impede their adoption on a large scale. We studied the performance of three public-key cryptosystems based on subset sum, learning with errors and ring learning with errors problems. We provide a systematic study for choosing their parameters to guarantee sufficient security levels and detail an asymptotic comparison between them in terms of storage and running time complexities. We accelerate the running time of these cryptosystems by exploiting the inherent parallelism in computations through a GPGPU-based parallel implementation. The cryptosystems are implemented using C++ on Intel(R) Xeon(R) multi-core 64-bit processors machine with CUDA-enabled Tesla K80 GPUs. The parallel implementation is based on OpenCL framework and can run on arbitrary hardware platform accelerators with minor changes. Several optimizations and efficient algorithms were used to compute the core operations in each cryptosystem to achieve optimum performance. The ring learning with errors based cryptosystem showed the best performance while the Subset Sum cryptosystem showed the highest speedup gain for the encryption primitive.

Parallelizing maximum likelihood classification (MLC) for supervised image classification by pipelined thread approach through high-level synthesis (HLS) on FPGA cluster

ABSTRACTHigh spectral, spatial, vertical and temporal resolution data are increasingly available and result in the serious challenge to process big remote-sensing images effectively and efficiently. This article introduced how to conduct supervised image classification by implementing maximum likelihood classification (MLC) over big image data on a field programmable gate array (FPGA) cloud. By comparing our prior work of implementing MLC on conventional cluster of multicore computers and graphics processing unit, it can be concluded that FPGAs can achieve the best performance in comparison to conventional CPU cluster and K40 GPU, and are more energy efficient. The proposed pipelined thread approach can be extended to other image-processing solutions to handle big data in the future.

GPU-Based Iterative Medical CT Image Reconstructions

The algebraic reconstruction technique (ART) is an iterative algorithm for CT (i.e., computed tomography) image reconstruction that delivers better image quality with less radiation dosage than the industry-standard filtered back projection (FBP). However, the high computational cost of ART requires researchers to turn to high-performance computing to accelerate the algorithm. Alas, existing approaches for ART suffer from inefficient design of compressed data structures and computational kernels on GPUs. Thus, this paper presents our CUDA-based CT image reconstruction tool based on the algebraic reconstruction technique (ART) or cuART. It delivers a compression and parallelization solution for ART-based image reconstruction on GPUs. We address the under-performing, but popular, GPU libraries, e.g., cuSPARSE, BRC, and CSR5, on the ART algorithm and propose a symmetry-based CSR format (SCSR) to further compress the CSR data structure and optimize data access for both SpMV and SpMV_T via a column-indices permutation. We also propose sorting-based global-level and sorting-free view-level blocking techniques to optimize the kernel computation by leveraging different sparsity patterns of the system matrix. The end result is that cuART can reduce the memory footprint significantly and enable practical CT datasets to fit into a single GPU. The experimental results on a NVIDIA Tesla K80 GPU illustrate that our approach can achieve up to 6.8x, 7.2x, and 5.4x speedups over counterparts that use cuSPARSE, BRC, and CSR5, respectively.

SwSpTRSV

Sparse triangular solve (SpTRSV) is one of the most important kernels in many real-world applications. Currently, much research on parallel SpTRSV focuses on level-set construction for reducing the number of inter-level synchronizations. However, the out-of-control data reuse and high cost for global memory or shared cache access in inter-level synchronization have been largely neglected in existing work. In this paper, we propose a novel data layout called Sparse Level Tile to make all data reuse under control, and design a Producer-Consumer pairing method to make any inter-level synchronization only happen in very fast register communication. We implement our data layout and algorithms on an SW26010 many-core processor, which is the main building-block of the current world fastest supercomputer Sunway Taihulight. The experimental results of testing all 2057 square matrices from the Florida Matrix Collection show that our method achieves an average speedup of 6.9 and the best speedup of 38.5 over parallel level-set method. Our method also outperforms the latest methods on a KNC many-core processor in 1856 matrices and the latest methods on a K80 GPU in 1672 matrices, respectively.

Simulating the behavior of the Human Brain on GPUs

The simulation of the behavior of the Human Brain is one of the most important challenges in computing today. The main problem consists of finding efficient ways to manipulate and compute the huge volume of data that this kind of simulations need, using the current technology. In this sense, this work is focused on one of the main steps of such simulation, which consists of computing the Voltage on neurons’ morphology. This is carried out using the Hines Algorithm and, although this algorithm is the optimum method in terms of number of operations, it is in need of non-trivial modifications to be efficiently parallelized on GPUs. We proposed several optimizations to accelerate this algorithm on GPU-based architectures, exploring the limitations of both, method and architecture, to be able to solve efficiently a high number of Hines systems (neurons). Each of the optimizations are deeply analyzed and described. Two different approaches are studied, one for mono-morphology simulations (batch of neurons with the same shape) and one for multi-morphology simulations (batch of neurons where every neuron has a different shape). In mono-morphology simulations we obtain a good performance using just a single kernel to compute all the neurons. However this turns out to be inefficient on multi-morphology simulations. Unlike the previous scenario, in multi-morphology simulations a much more complex implementation is necessary to obtain a good performance. In this case, we must execute more than one single GPU kernel. In every execution (kernel call) one specific part of the batch of the neurons is solved. These parts can be seen as multiple and independent tridiagonal systems. Although the present paper is focused on the simulation of the behavior of the Human Brain, some of these techniques, in particular those related to the solving of tridiagonal systems, can be also used for multiple oil and gas simulations. Our studies have proven that the optimizations proposed in the present work can achieve high performance on those computations with a high number of neurons, being our GPU implementations about 4× and 8× faster than the OpenMP multicore implementation (16 cores), using one and two NVIDIA K80 GPUs respectively. Also, it is important to highlight that these optimizations can continue scaling, even when dealing with a very high number of neurons.

Multigroup Monte Carlo on GPUs: Comparison of history- and event-based algorithms

This paper presents an investigation of the performance of different multigroup Monte Carlo transport algorithms on GPUs with a discussion of both history-based and event-based approaches. Several algorithmic improvements are introduced for both approaches. By modifying the history-based algorithm that is traditionally favored in CPU-based MC codes to occasionally filter out dead particles to reduce thread divergence, performance exceeds that of either the pure history-based or event-based approaches. The impacts of several algorithmic choices are discussed, including performance studies on Kepler and Pascal generation NVIDIA GPUs for fixed source and eigenvalue calculations. Single-device performance equivalent to 20–40 CPU cores on the K40 GPU and 60–80 CPU cores on the P100 GPU is achieved. In addition, nearly perfect multi-device parallel weak scaling is demonstrated on more than 16,000 nodes of the Titan supercomputer.

Real-time parallel image processing applications on multicore CPUs with OpenMP and GPGPU with CUDA

This paper presents real-time image processing applications using multicore and multiprocessing technologies. To this end, parallel image segmentation was performed on many images covering the entire surface of the same metallic and cylindrical moving objects. Experimental results on multicore CPU with OpenMP platform showed that by increasing the chunk size, the execution time decreases approximately four times in comparison with serial computing. The same experiments were implemented on GPGPU using four techniques: (1) Single image transmission with single pixel processing; (2) Single image transmission with multiple pixel processing; (3) Multiple image transmission with single pixel processing; and (4) Multiple image transmission with multiple pixel processing. All techniques were implemented on GeForce, Tesla K20 and Tesla K40. Experimental results of GPU with CUDA platform showed that by increasing the core number speedup is increased. Tesla K40 gave the best results of 35 and 12 (for the first technique), 36 and 13 (for the second technique), 54 and 16 (for the third technique), 71 and 17 (for the fourth technique) times improvement without and with data transmission time in comparison with serial computing. As a result, users are suggested to use Tesla K40 GPU and Multiple image transmission with multiple pixel processing to get the maximum performance.

Realization of software based imaging energy analyzer on GPU

Here, the realization of the Filtering the Effect of Chromatic Aberration (FECA) method is presented as Software Based Imaging Energy Analyzer (SBIEA) on an NVIDA Tesla K40 GPU. This method makes possible the correction of chromatic aberration and, therefore, spectrometry, as well as, real- and k-space monochromatic imaging by rapidly calculated spectral image sequences of monochromatic images in light and charged particle optics without the application of additional energy filters. The GPU accelerated realization of this method provides a good solution with reasonable calculation time on large size image sequences and opens new direction toward the better quality monochromatic real- and k-space imaging, as well as, the developments of software based instrumentation. The presented GPU accelerated realization of the FECA-SBIEA method provides promising alternative solution for making fast and accurate monochromatic spectral imaging that can be better than the usually applied electrostatic and magnetic analyzers due to the overlapping effect of different energy images especially in the case of large acceptance angle optics.