NVIDIA K80 Research Articles

DjiNN and Tonic

As applications such as Apple Siri, Google Now, Microsoft Cortana, and Amazon Echo continue to gain traction, web-service companies are adopting large deep neural networks (DNN) for machine learning challenges such as image processing, speech recognition, natural language processing, among others. A number of open questions arise as to the design of a server platform specialized for DNN and how modern warehouse scale computers (WSCs) should be outfitted to provide DNN as a service for these applications. In this paper, we present DjiNN , an open infrastructure for DNN as a service in WSCs, and Tonic Suite , a suite of 7 end-to-end applications that span image, speech, and language processing. We use DjiNN to design a high throughput DNN system based on massive GPU server designs and provide insights as to the varying characteristics across applications. After studying the throughput, bandwidth, and power properties of DjiNN and Tonic Suite, we investigate several design points for future WSC architectures. We investigate the total cost of ownership implications of having a WSC with a disaggregated GPU pool versus a WSC composed of homogeneous integrated GPU servers. We improve DNN throughput by over 120x for all but one application (40x for Facial Recognition) on an NVIDIA K40 GPU. On a GPU server composed of 8 NVIDIA K40s, we achieve near-linear scaling (around 1000x throughput improvement) for 3 of the 7 applications. Through our analysis, we also find that GPU-enabled WSCs improve total cost of ownership over CPU-only designs by 4-20x, depending on the composition of the workload

RETRACTED: Batched matrix computations on hardware accelerators based on GPUs

Scientific applications require solvers that work on many small size problems that are independent from each other. At the same time, the high-end hardware evolves rapidly and becomes ever more throughput-oriented and thus there is an increasing need for an effective approach to develop energy-efficient, high-performance codes for these small matrix problems that we call batched factorizations. The many applications that need this functionality could especially benefit from the use of GPUs, which currently are four to five times more energy efficient than multicore CPUs on important scientific workloads. This paper, consequently, describes the development of the most common, one-sided factorizations, Cholesky, LU, and QR, for a set of small dense matrices. The algorithms we present together with their implementations are, by design, inherently parallel. In particular, our approach is based on representing the process as a sequence of batched BLAS routines that are executed entirely on a GPU. Importantly, this is unlike the LAPACK and the hybrid MAGMA factorization algorithms that work under drastically different assumptions of hardware design and efficiency of execution of the various computational kernels involved in the implementation. Thus, our approach is more efficient than what works for a combination of multicore CPUs and GPUs for the problems sizes of interest of the application use cases. The paradigm where upon a single chip (a GPU or a CPU) factorizes a single problem at a time is not at all efficient in our applications’ context. We illustrate all of these claims through a detailed performance analysis. With the help of profiling and tracing tools, we guide our development of batched factorizations to achieve up to two-fold speedup and three-fold better energy efficiency as compared against our highly optimized batched CPU implementations based on MKL library. The tested system featured two sockets of Intel Sandy Bridge CPUs and we compared with a batched LU factorizations featured in the CUBLAS library for GPUs, we achieve as high as 2.5× speedup on the NVIDIA K40 GPU.

Cache Locality-Centric Parallel String Matching on Many-Core Accelerator Chips

Aho-Corasick (AC) algorithm is a multiple patterns string matching algorithm commonly used in computer and network security and bioinformatics, among many others. In order to meet the highly demanding computational requirements imposed on these applications, achieving high performance for the AC algorithm is crucial. In this paper, we present a high performance parallelization of the AC on the many-core accelerator chips such as the Graphic Processing Unit (GPU) from Nvidia and the Intel Xeon Phi. Our parallelization approach significantly improves the cache locality of the AC by partitioning a given set of string patterns into multiple smaller sets of patterns in a space-efficient way. Using the multiple pattern sets, intensive pattern matching operations are concurrently conducted with respect to the whole input text data. Compared with the previous approaches where the input data is partitioned amongst multiple threads instead of partitioning the pattern set, our approach significantly improves the performance. Experimental results show that our approach leads up to 2.73 times speedup on the Nvidia K20 GPU and 2.00 times speedup on the Intel Xeon Phi compared with the previous approach. Our parallel implementation delivers up to 693 Gbps throughput performance on the K20.

GPU-Accelerated Longwave Radiation Scheme of the Rapid Radiative Transfer Model for General Circulation Models (RRTMG)

Atmospheric radiative transfer models calculate radiative transfer of electromagnetic radiation through a planetary atmosphere. One of such models is the rapid radiative transfer model (RRTM), which evaluates longwave and shortwave atmospheric radiative fluxes and heating rates. The RRTM for general circulation models (GCMs), RRTMG, is an accelerated version based on the single-column reference of RRTM. The longwave radiation scheme of RRTM for GCMs (RRTMG_LW) is one model that utilizes the correlated-k approach to calculate longwave fluxes and heating rates for application to GCMs. In this paper, the feasibility of using graphics processing units (GPUs) to accelerate the in weather research and forecasting (WRF) model is examined. GPUs allow a substantial performance improvement in RRTMG_LW with a large number of parallel compute cores at low cost and power. Our GPU version of RRTMG_LW yields the bit-exact outputs as its original Fortran code. Our results show that NVIDIA's K40 GPU achieves a speedup of x as compared to its CPU counterpart running on one CPU core of Intel Xeon E5-2603, whereas the speedup for one CPU socket (4 cores) of the Xeon E5-2603 with respect to one CPU core is only 3.2×.

Simulation of disc-bulge-halo galaxies using parallel GPU based codes

AbstractWe compare the performance of the very popular Tree-GPU code BONSAI with the older Particle-(Multi)Mesh code SUPERBOX. Both code we run on a same hardware using the GPU acceleration for the force calculation. SUPERBOX is a particle-mesh code with high resolution sub-grid and a higher order NGP (nearest grid point) force-calculation scheme. In our research, we are aiming to demonstrate that the new parallel version of SUPERBOX is capable to do the high resolution simulations of the interaction of the system of disc-bulge-halo composed galaxy. We describe the improvement of performance and scalability of SUPERBOX particularly for the Kepler cluster (NVIDIA K20 GPU). A comparison was made with the very popular and publicly available Tree-GPU code BONSAI†.