NVIDIA CUDA Toolkit Research Articles

CAD-ASTRA: a versatile and efficient mesh projector for X-ray tomography with the ASTRA-toolbox.

Accurate and fast simulation of X-ray projection data from mesh models has many applications in academia and industry, ranging from 3D X-ray computed tomography (XCT) reconstruction algorithms to radiograph-based object inspection and quality control. While software tools for the simulation of X-ray projection data from mesh models are available, they lack either performance, public availability, flexibility to implement non-standard scanning geometries, or easy integration with existing 3D XCT software. In this paper, we propose CAD-ASTRA, a highly versatile toolbox for fast simulation of X-ray projection data from mesh models. While fully functional as standalone software, it is also compatible with the ASTRA toolbox, an open-source toolbox for flexible tomographic reconstruction. CAD-ASTRA provides three specialized GPU projectors based on state-of-the-art algorithms for 3D rendering, implemented using the NVIDIA CUDA Toolkit and the OptiX engine. First, it enables X-ray phase contrast simulations by modeling refraction through ray tracing. Second, it allows the back-propagation of projective errors to mesh vertices, enabling immediate application in mesh reconstruction, deep learning, and other optimization routines. Finally, CAD-ASTRA allows simulation of polychromatic X-ray projections from heterogeneous objects with a source of finite focal spot size. Use cases on a CAD-based inspection task, a phase contrast experiment, a combined mesh-volumetric data projection, and a mesh reconstruction demonstrate the wide applicability of CAD-ASTRA.

Spinsim: A GPU optimized python package for simulating spin-half and spin-one quantum systems

Spinsim simulates the quantum dynamics of unentangled spin-1/2 and spin-1 systems evolving under time-dependent control. While other solvers for the time-dependent Schrödinger equation optimize for larger state spaces but less temporally-rich control, spinsim is optimized for intricate time evolution of a minimalist system. Efficient simulation of individual or ensemble quanta driven by adiabatic sweeps, elaborate pulse sequences, complex signals and non-Gaussian noise is the primary target application. We achieve fast and robust evolution using a geometric integrator to bound errors over many steps, and split the calculation parallel-in-time on a GPU using the numba just-in-time compiler. Speed-up is three orders of magnitude over QuTip's sesolve and Mathematica's NDSolve, and four orders over SciPy's ivp_solve for equal accuracy. Interfaced through python, spinsim should be useful for simulating robust state preparation, inversion and dynamical decoupling sequences in NMR and MRI, and in quantum control, memory and sensing applications with two- and three-level quanta. Program summaryProgram Title: SpinsimCPC Library link to program files:https://doi.org/10.17632/f6rdk4gyxr.1Developer's repository link:https://github.com/alexander-tritt-monash/spinsimLicensing provisions: BSD 3-clauseProgramming language: Python (3.7 or greater)Nature of problem: Quantum sensing is a domain of quantum technology where the dynamics of quantum systems are used to infer properties of the systems' environments. The development of quantum sensing protocols is greatly sped-up by software simulations of the new techniques. Quantum sensing simulation benefits from temporally-rich control of individual quanta. However, current specialized time-dependent Schrödinger equation solvers are instead optimized only for simple pulses in large Hilbert spaces. Thus, there is a need for efficient simulation of individual or ensemble quanta driven by adiabatic sweeps, elaborate pulse sequences, complex signals and non-Gaussian noise.Solution method: Spinsim simulates the quantum dynamics of spin-1/2 and spin-1 systems evolving under time-dependent control. We first speed up the integration of the time-dependent Schrödinger equation by splitting the calculation parallel-in-time on a GPU using the numba [1] just-in-time compiler. We achieve fast and robust evolution using a geometric integrator to bound errors over many steps. A dynamic rotating frame transformation and Lie-Trotter decomposition are used to decrease effective step sizes on long and short time scales, respectively. Hence, each individual step is more accurate. Spinsim is interfaced via a python package, meaning it can be used by researchers inexperienced with geometric integrators and GPU parallelism.Additional comments including restrictions and unusual features: To use the (default) Nvidia cuda GPU parallelization, one needs to have a cuda compatible Nvidia GPU [2]. For cuda mode to function, one also needs to install the Nvidia cuda toolkit [3]. If cuda is not available on the system, the simulator will automatically parallelize over multicore CPUs instead. Developed and tested on Windows 10; tested on Windows 11. CPU functionality tested on MacOS 10.16 Big Sur (note that MacOS 10.14 Mojave and higher are not compatible with cuda hardware and software). The package (including cuda functionality) is in principle compatible with Linux, but functionality has not been tested.

BloSM: Blockchain-Based Service Migration for Connected Cars in Embedded Edge Environment

Edge computing represents the future of computing paradigms that perform tasks near the user plane. The integration of blockchain with edge computing provides the added advantage of secured and trusted communication. In this paper, we propose a blockchain-based service migration by developing edge clusters using NVIDIA Jetson boards in an embedded edge environment, using containers and Kubernetes as a container orchestration capable of handling real-time computation-intensive deep learning tasks. Resource constraints in the edge and client movement are the proposed scenarios for service migration. Container migration due to mobile clients is integrated with blockchain to find a suitable destination, meta-based node evaluation, and secured data transfer in the connected car environment. Each service request migration takes, on average, 361 ms. The employed container migration method takes 75.11 s and 70.46 s to migrate application containers that use NVIDIA CUDA Toolkit. Finally, we evaluate the efficiency of blockchain to find the destination node through performance parameters such as latency, throughput, storage, and bandwidth.

Modeling and analyzing evaluation cost of CUDA kernels

General-purpose programming on GPUs (GPGPU) is becoming increasingly in vogue as applications such as machine learning and scientific computing demand high throughput in vector-parallel applications. NVIDIA's CUDA toolkit seeks to make GPGPU programming accessible by allowing programmers to write GPU functions, called kernels, in a small extension of C/C++. However, due to CUDA's complex execution model, the performance characteristics of CUDA kernels are difficult to predict, especially for novice programmers. This paper introduces a novel quantitative program logic for CUDA kernels, which allows programmers to reason about both functional correctness and resource usage of CUDA kernels, paying particular attention to a set of common but CUDA-specific performance bottlenecks. The logic is proved sound with respect to a novel operational cost semantics for CUDA kernels. The semantics, logic and soundness proofs are formalized in Coq. An inference algorithm based on LP solving automatically synthesizes symbolic resource bounds by generating derivations in the logic. This algorithm is the basis of RaCuda, an end-to-end resource-analysis tool for kernels, which has been implemented using an existing resource-analysis tool for imperative programs. An experimental evaluation on a suite of CUDA benchmarks shows that the analysis is effective in aiding the detection of performance bugs in CUDA kernels.

Effective Extensible Programming: Unleashing Julia on GPUs

GPUs and other accelerators are popular devices for accelerating compute-intensive, parallelizable applications. However, programming these devices is a difficult task. Writing efficient device code is challenging, and is typically done in a low-level programming language. High-level languages are rarely supported, or do not integrate with the rest of the high-level language ecosystem. To overcome this, we propose compiler infrastructure to efficiently add support for new hardware or environments to an existing programming language. We evaluate our approach by adding support for NVIDIA GPUs to the Julia programming language. By integrating with the existing compiler, we significantly lower the cost to implement and maintain the new compiler, and facilitate reuse of existing application code. Moreover, use of the high-level Julia programming language enables new and dynamic approaches for GPU programming. This greatly improves programmer productivity, while maintaining application performance similar to that of the official NVIDIA CUDA toolkit.

Massively parallel multicanonical simulations

Generalized-ensemble Monte Carlo simulations such as the multicanonical method and similar techniques are among the most efficient approaches for simulations of systems undergoing discontinuous phase transitions or with rugged free-energy landscapes. As Markov chain methods, they are inherently serial computationally. It was demonstrated recently, however, that a combination of independent simulations that communicate weight updates at variable intervals allows for the efficient utilization of parallel computational resources for multicanonical simulations. Implementing this approach for the many-thread architecture provided by current generations of graphics processing units (GPUs), we show how it can be efficiently employed with of the order of 104 parallel walkers and beyond, thus constituting a versatile tool for Monte Carlo simulations in the era of massively parallel computing. We provide the fully documented source code for the approach applied to the paradigmatic example of the two-dimensional Ising model as starting point and reference for practitioners in the field. Program summaryProgram Title: cudamucaProgram Files doi:http://dx.doi.org/10.17632/tzhfpdymv9.1Licensing provisions: Creative Commons Attribution license (CC BY 4.0)Programming language: C++, CUDAExternal routines/libraries: NVIDIA CUDA Toolkit 6.5 or newerNature of problem: The program determines weights for a multicanonical simulation of the 2D Ising model to result in a flat energy histogram. A final production run with these weights provides an estimate of the density of states of the model.Solution method: The code uses a parallel variant of the multicanonical method employing many parallel walkers that accumulate a common histogram. The resulting histogram is used to determine the weight function for the next iteration. Once the iteration has converged, simulations visit all possible energies with the same probability.Additional comments including restrictions and unusual features: The system size and size of the population of replicas are limited depending on the memory of the GPU device used. Code repository at https://github.com/CQT-Leipzig/cudamuca.

GPU accelerated population annealing algorithm

Population annealing is a promising recent approach for Monte Carlo simulations in statistical physics, in particular for the simulation of systems with complex free-energy landscapes. It is a hybrid method, combining importance sampling through Markov chains with elements of sequential Monte Carlo in the form of population control. While it appears to provide algorithmic capabilities for the simulation of such systems that are roughly comparable to those of more established approaches such as parallel tempering, it is intrinsically much more suitable for massively parallel computing. Here, we tap into this structural advantage and present a highly optimized implementation of the population annealing algorithm on GPUs that promises speed-ups of several orders of magnitude as compared to a serial implementation on CPUs. While the sample code is for simulations of the 2D ferromagnetic Ising model, it should be easily adapted for simulations of other spin models, including disordered systems. Our code includes implementations of some advanced algorithmic features that have only recently been suggested, namely the automatic adaptation of temperature steps and a multi-histogram analysis of the data at different temperatures. Program summaryProgram Title: PAIsingProgram Files doi:http://dx.doi.org/10.17632/sgzt4b7b3m.1Licensing provisions: Creative Commons Attribution license (CC BY 4.0)Programming language: C, CUDAExternal routines/libraries: NVIDIA CUDA Toolkit 6.5 or newerNature of problem: The program calculates the internal energy, specific heat, several magnetization moments, entropy and free energy of the 2D Ising model on square lattices of edge length L with periodic boundary conditions as a function of inverse temperature β.Solution method: The code uses population annealing, a hybrid method combining Markov chain updates with population control. The code is implemented for NVIDIA GPUs using the CUDA language and employs advanced techniques such as multi-spin coding, adaptive temperature steps and multi-histogram reweighting.Additional comments: Code repository at https://github.com/LevBarash/PAising.The system size and size of the population of replicas are limited depending on the memory of the GPU device used.For the default parameter values used in the sample programs, L=64, θ=100, β0=0, βf=1, Δβ=0.005, R=20000, a typical run time on an NVIDIA Tesla K80 GPU is 151 seconds for the single spin coded (SSC) and 17 seconds for the multi-spin coded (MSC) program (see Section 2 for a description of these parameters).

CUDA programs for solving the time-dependent dipolar Gross–Pitaevskii equation in an anisotropic trap

In this paper we present new versions of previously published numerical programs for solving the dipolar Gross-Pitaevskii (GP) equation including the contact interaction in two and three spatial dimensions in imaginary and in real time, yielding both stationary and non-stationary solutions. New versions of programs were developed using CUDA toolkit and can make use of Nvidia GPU devices. The algorithm used is the same split-step semi-implicit Crank-Nicolson method as in the previous version (R. Kishor Kumar et al., Comput. Phys. Commun. 195, 117 (2015)), which is here implemented as a series of CUDA kernels that compute the solution on the GPU. In addition, the Fast Fourier Transform (FFT) library used in the previous version is replaced by cuFFT library, which works on CUDA-enabled GPUs. We present speedup test results obtained using new versions of programs and demonstrate an average speedup of 12 to 25, depending on the program and input size.

PRAND: GPU accelerated parallel random number generation library: Using most reliable algorithms and applying parallelism of modern GPUs and CPUs

The library PRAND for pseudorandom number generation for modern CPUs and GPUs is presented. It contains both single-threaded and multi-threaded realizations of a number of modern and most reliable generators recently proposed and studied in Barash (2011), Matsumoto and Tishimura (1998), L’Ecuyer (1999,1999), Barash and Shchur (2006) and the efficient SIMD realizations proposed in Barash and Shchur (2011). One of the useful features for using PRAND in parallel simulations is the ability to initialize up to 1019 independent streams. Using massive parallelism of modern GPUs and SIMD parallelism of modern CPUs substantially improves performance of the generators. Program summaryProgram title: PRANDCatalogue identifier: AESB_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AESB_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 45979No. of bytes in distributed program, including test data, etc.: 23953564Distribution format: tar.gzProgramming language: Cuda C, Fortran.Computer: PC, workstation, laptop, or server with NVIDIA GPU (tested on Tesla X2070, Fermi C2050, GeForce GT540M) and with Intel or AMD processor.Operating system: Linux with CUDA version 5.0 or later. Should also run on MacOs, Windows, or UNIX.RAM: 4 MbytesClassification: 4.13.Nature of problem:Any calculation requiring uniform pseudorandom number generator, in particular, Monte Carlo calculations. Any calculation or simulation requiring uncorrelated parallel streams of uniform pseudorandom numbers.Solution method:The library contains realization of a number of modern and reliable generators: MT19937, MRG32K3A and LFSR113. Also new realizations of the method based on parallel evolution of an ensemble of transformations of two-dimensional torus are included in the library: GM19, GM29, GM31, GM61, GM55, GQ58.1, GQ58.3 and GQ58.4. The library contains: single-threaded and multi-threaded realizations for GPU, single-threaded realizations for CPU, realizations for CPU based on SSE command set. Also, the library contains the abilities to jump ahead inside RNG sequence and to initialize independent random number streams with block splitting method for each of the RNGs.Restrictions:Nvidia Cuda Toolkit version 5.0 or later should be installed. To use GPU realizations, Nvidia GPU supporting CUDA and the corresponding Nvidia driver should be installed. For SSE realizations of the generators, Intel or AMD CPU supporting SSE2 command set is required. In order to use the SSE realization of LFSR113, CPU must support SSE4 command set.Additional comments:A version of this program, which only contains the realizations for CPUs, is held in the Library as Catalog Id., AEIT_v2_0 (RNGSSELIB). It does not require a GPU device or CUDA compiler.Running time:The tests and the examples included in the package take less or about one minute to run. Running time is analyzed in detail in Section 8 of the paper.

CUDA programs for the GPU computing of the Swendsen–Wang multi-cluster spin flip algorithm: 2D and 3D Ising, Potts, and XY models

We present sample CUDA programs for the GPU computing of the Swendsen–Wang multi-cluster spin flip algorithm. We deal with the classical spin models; the Ising model, the q-state Potts model, and the classical XY model. As for the lattice, both the 2D (square) lattice and the 3D (simple cubic) lattice are treated. We already reported the idea of the GPU implementation for 2D models (Komura and Okabe, 2012). We here explain the details of sample programs, and discuss the performance of the present GPU implementation for the 3D Ising and XY models. We also show the calculated results of the moment ratio for these models, and discuss phase transitions. Program summaryProgram title: SWspinCatalogue identifier: AERM_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AERM_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 5632No. of bytes in distributed program, including test data, etc.: 14688Distribution format: tar.gzProgramming language: C, CUDA.Computer: System with an NVIDIA CUDA enabled GPU.Operating system: System with an NVIDIA CUDA enabled GPU.Classification: 23.External routines: NVIDIA CUDA Toolkit 3.0 or newerNature of problem:Monte Carlo simulation of classical spin systems. Ising, q-state Potts model, and the classical XY model are treated for both two-dimensional and three-dimensional lattices.Solution method:GPU-based Swendsen–Wang multi-cluster spin flip Monte Carlo method. The CUDA implementation for the cluster-labeling is based on the work by Hawick et al. [1] and that by Kalentev et al. [2].Restrictions:The system size is limited depending on the memory of a GPU.Running time:For the parameters used in the sample programs, it takes about a minute for each program. Of course, it depends on the system size, the number of Monte Carlo steps, etc.

GPUmotif: An Ultra-Fast and Energy-Efficient Motif Analysis Program Using Graphics Processing Units

Computational detection of TF binding patterns has become an indispensable tool in functional genomics research. With the rapid advance of new sequencing technologies, large amounts of protein-DNA interaction data have been produced. Analyzing this data can provide substantial insight into the mechanisms of transcriptional regulation. However, the massive amount of sequence data presents daunting challenges. In our previous work, we have developed a novel algorithm called Hybrid Motif Sampler (HMS) that enables more scalable and accurate motif analysis. Despite much improvement, HMS is still time-consuming due to the requirement to calculate matching probabilities position-by-position. Using the NVIDIA CUDA toolkit, we developed a graphics processing unit (GPU)-accelerated motif analysis program named GPUmotif. We proposed a “fragmentation" technique to hide data transfer time between memories. Performance comparison studies showed that commonly-used model-based motif scan and de novo motif finding procedures such as HMS can be dramatically accelerated when running GPUmotif on NVIDIA graphics cards. As a result, energy consumption can also be greatly reduced when running motif analysis using GPUmotif. The GPUmotif program is freely available at http://sourceforge.net/projects/gpumotif/

GPU-based four-dimensional general-relativistic ray tracing

This paper presents a new general-relativistic ray tracer that enables image synthesis on an interactive basis by exploiting the performance of graphics processing units (GPUs). The application is capable of visualizing the distortion of the stellar background as well as trajectories of moving astronomical objects orbiting a compact mass. Its source code includes metric definitions for the Schwarzschild and Kerr spacetimes that can be easily extended to other metric definitions, relying on its object-oriented design. The basic functionality features a scene description interface based on the scripting language Lua, real-time image output, and the ability to edit almost every parameter at runtime. The ray tracing code itself is implemented for parallel execution on the GPU using NVidia’s Compute Unified Device Architecture (CUDA), which leads to performance improvement of an order of magnitude compared to a single CPU and makes the application competitive with small CPU cluster architectures.Program summaryProgram title: GpuRay4DCatalog identifier: AEMV_v1_0Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEMV_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 73649No. of bytes in distributed program, including test data, etc.: 1334251Distribution format: tar.gzProgramming language: C++, CUDA.Computer: Linux platforms with a NVidia CUDA enabled GPU (Compute Capability 1.3 or higher), C++ compiler, NVCC (The CUDA Compiler Driver).Operating system: Linux.RAM: 2 GBClassification: 1.5.External routines: OpenGL Utility Toolkit development files, NVidia CUDAToolkit 3.2, Lua5.2Nature of problem: Ray tracing in four-dimensional Lorentzian spacetimes.Solution method: Numerical integration of light rays, GPU-based parallel programming using CUDA, 3D-Rendering via OpenGL.Running time: Problem dependent, several seconds up to hours.

Accelerating numerical solution of stochastic differential equations with CUDA

Numerical integration of stochastic differential equations is commonly used in many branches of science. In this paper we present how to accelerate this kind of numerical calculations with popular NVIDIA Graphics Processing Units using the CUDA programming environment. We address general aspects of numerical programming on stream processors and illustrate them by two examples: the noisy phase dynamics in a Josephson junction and the noisy Kuramoto model. In presented cases the measured speedup can be as high as 675× compared to a typical CPU, which corresponds to several billion integration steps per second. This means that calculations which took weeks can now be completed in less than one hour. This brings stochastic simulation to a completely new level, opening for research a whole new range of problems which can now be solved interactively. Program summary Program title: SDE Catalogue identifier: AEFG_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEFG_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Gnu GPL v3 No. of lines in distributed program, including test data, etc.: 978 No. of bytes in distributed program, including test data, etc.: 5905 Distribution format: tar.gz Programming language: CUDA C Computer: any system with a CUDA-compatible GPU Operating system: Linux RAM: 64 MB of GPU memory Classification: 4.3 External routines: The program requires the NVIDIA CUDA Toolkit Version 2.0 or newer and the GNU Scientific Library v1.0 or newer. Optionally gnuplot is recommended for quick visualization of the results. Nature of problem: Direct numerical integration of stochastic differential equations is a computationally intensive problem, due to the necessity of calculating multiple independent realizations of the system. We exploit the inherent parallelism of this problem and perform the calculations on GPUs using the CUDA programming environment. The GPU's ability to execute hundreds of threads simultaneously makes it possible to speed up the computation by over two orders of magnitude, compared to a typical modern CPU. Solution method: The stochastic Runge–Kutta method of the second order is applied to integrate the equation of motion. Ensemble-averaged quantities of interest are obtained through averaging over multiple independent realizations of the system. Unusual features: The numerical solution of the stochastic differential equations in question is performed on a GPU using the CUDA environment. Running time: < 1 minute

Long-range interactions in a lattice gas automaton model for immiscible fluids

Generalized-ensemble Monte Carlo simulations such as the multicanonical method and similar techniques are among the most efficient approaches for simulations of systems undergoing discontinuous phase transitions or with rugged free-energy landscapes. As Markov chain methods, they are inherently serial computationally. It was demonstrated recently, however, that a combination of independent simulations that communicate weight updates at variable intervals allows for the efficient utilization of parallel computational resources for multicanonical simulations. Implementing this approach for the many-thread architecture provided by current generations of graphics processing units (GPUs), we show how it can be efficiently employed with of the order of 104 parallel walkers and beyond, thus constituting a versatile tool for Monte Carlo simulations in the era of massively parallel computing. We provide the fully documented source code for the approach applied to the paradigmatic example of the two-dimensional Ising model as starting point and reference for practitioners in the field.Program Title: cudamucaProgram Files doi: http://dx.doi.org/10.17632/tzhfpdymv9.1 Licensing provisions: Creative Commons Attribution license (CC BY 4.0)Programming language: C++, CUDAExternal routines/libraries: NVIDIA CUDA Toolkit 6.5 or newerNature of problem: The program determines weights for a multicanonical simulation of the 2D Ising model to result in a flat energy histogram. A final production run with these weights provides an estimate of the density of states of the model.Solution method: The code uses a parallel variant of the multicanonical method employing many parallel walkers that accumulate a common histogram. The resulting histogram is used to determine the weight function for the next iteration. Once the iteration has converged, simulations visit all possible energies with the same probability.Additional comments including restrictions and unusual features: The system size and size of the population of replicas are limited depending on the memory of the GPU device used. Code repository at https://github.com/CQT-Leipzig/cudamuca.