Chapter 6 - GPU programming with CUDA

Abstract

Graphics processing units or GPUs are being widely used for general purpose programming. CUDA provides a collection of modifications to the C++ compiler and a library of functions that can be can be used for general purpose programming of Nvidia GPUs. GPUs employ features of both SIMD and MIMD processors. GPUs are not ordinarily standalone processors; rather a typical GPU is paired with a CPU, which carries out basic functions such as I/O, memory allocation, and initialization. We develop a range of CUDA programs from a basic “hello, world” program to programs for numerical integration to programs for sorting. We illustrate the basics of writing CUDA kernels, which are functions that run on the GPU but are started by the host processor—the CPU. We make use of CUDA global and shared memory, barriers, and warp shuffles to accelerate the performance of our programs and to address race conditions.