Thread Divergence Research Articles

A software-based dynamic-warp scheduling approach for load-balancing the Viola–Jones face detection algorithm on GPUs

Face detection is a key component in applications such as security surveillance and human–computer interaction systems, and real-time recognition is essential in many scenarios. The Viola–Jones algorithm is an attractive means of meeting the real time requirement, and has been widely implemented on custom hardware, FPGAs and GPUs. We demonstrate a GPU implementation that achieves competitive performance, but with low development costs. Our solution treats the irregularity inherent to the algorithm using a novel dynamic warp scheduling approach that eliminates thread divergence. This new scheme also employs a thread pool mechanism, which significantly alleviates the cost of creating, switching, and terminating threads. Compared to static thread scheduling, our dynamic warp scheduling approach reduces the execution time by a factor of 3. To maximize detection throughput, we also run on multiple GPUs, realizing 95.6 FPS on 5 Fermi GPUs.

Operator-level GPU-Accelerated Branch and Bound Algorithms

Branch-and-Bound (B&B) algorithms are well-known tree-based exploratory methods for solving to optimality NP-hard discrete optimization problems. The construction of the B&B tree and its exploration are performed using four operators: branching, bounding, selection and pruning. Such algorithms are irregular which makes challenging their parallel design and implementation on GPU accelerators. Among the few existing related works, we have recently revisited on GPU the bounding operator. The reported results show that speedups up to × 100 can be obtained on recent GPU cards. In this paper, we address the GPU-based design and implementation of B&B algorithms considering the branching and pruning operators as well as the bounding one. The proposed template transforms the unpredictable and irregular workload associated to the explored B&B tree into regular data-parallel kernels optimized for the SIMD-based execution model of GPUs. Thread divergence and uncoalesced memory accesses are considered in the optimization process. The proposed approach has been experimented on the Flow-Shop scheduling problem and compared to another GPU-based strategy and to a cluster of workstations (COWs) based approach. The reported results demonstrate the efficiency of the proposed approach over the two other ones. Speedups up to × 160 are obtained for large problem instances using an Nvidia Tesla C2050 hardware configuration.

Reducing thread divergence in a GPU‐accelerated branch‐and‐bound algorithm

SUMMARYIn this paper, we address the design and implementation of graphical processing unit (GPU)‐accelerated branch‐and‐bound algorithms (B&B) for solving flow‐shop scheduling optimization problems (FSP). Such applications are CPU‐time consuming and highly irregular. On the other hand, GPUs are massively multithreaded accelerators using the single instruction multiple data model at execution. A major issue that arises when executing on GPU, a B&B applied to FSP is thread or branch divergence. Such divergence is caused by the lower bound function of FSP that contains many irregular loops and conditional instructions. Our challenge is therefore to revisit the design and implementation of B&B applied to FSP dealing with thread divergence. Extensive experiments of the proposed approach have been carried out on well‐known FSP benchmarks using an Nvidia Tesla (C2050 GPU card (http://www.nvidia.com/docs/IO/43395/NV_DS_Tesla_C2050_C2070_jul10_lores.pdf)). Compared with a CPU‐based execution, accelerations up to × 77.46 are achieved for large problem instances. Copyright © 2012 John Wiley & Sons, Ltd.

SU-E-T-476: GPU-Based Monte Carlo Radiotherapy Dose Calculation Using Phase- Space Sources.

To design an efficient method for utilizing phase-space source models in the GPU-based Monte Carlo (MC) dose calculation engine gDPM. In GPU-based MC algorithms, particles are transported in parallel on different threads. Particles of different types and energies can require significantly different execution times. This can cause "thread divergence" and lower efficiency when source particles are read sequentially from a phase-space file. We have developed a strategy for utilizing phase- space files in a GPU compatible manner whereby the particles are grouped into phase-space-lets (PSLs) by type, energy, and location in the phase- space plane. This allows for dose calculations using only particles inside the field opening defined by the secondary collimators. For validation, the gDPM PSL implementation is compared with DOSXYZnrc using a BEAMnrc phase-space source model as input. Two phase-spaces were generated using a BEAMnrc head model of a 6MV Varian Clinac 21EX, one above the upper jaws used to generate PSLs for gDPM and the other below the lower jaws used for DOSXYZnrc dose calculation. Profiles and depth dose curves for a variety of field sizes were generated in a water phantom. The agreement between gDPM and DOSXYZnrc is within 2% for all field sizes. For the 10 cm × 10 cm field, the calculation times of 650 million histories were 147 CPU hours and 54 GPU seconds for DOSXYZnrc and gDPM, respectively. In addition, we have tested the gDPM PSL implementation for dose calculation in a realistic 7-field IMRT tongue treatment plan. The calculation times were 59 CPU-hours and 66 GPU- seconds for DOSXYZnrc and gDPM for 485 million histories, respectively. Gamma pass rate for the two dose distributions was 99.54% for 3 mm/3% criteria within the 10% isodose. Methods for the efficient use of phase-space sources for GPU-based MC dose calculations have been developed.

Simultaneous branch and warp interweaving for sustained GPU performance

Single-Instruction Multiple-Thread (SIMT) micro-architectures implemented in Graphics Processing Units (GPUs) run fine-grained threads in lockstep by grouping them into units, referred to as warps, to amortize the cost of instruction fetch, decode and control logic over multiple execution units. As individual threads take divergent execution paths, their processing takes place sequentially, defeating part of the efficiency advantage of SIMD execution. We present two complementary techniques that mitigate the impact of thread divergence on SIMT micro-architectures. Both techniques relax the SIMD execution model by allowing two distinct instructions to be scheduled to disjoint subsets of the the same row of execution units, instead of one single instruction. They increase flexibility by providing more thread grouping opportunities than SIMD, while preserving the affinity between threads to avoid introducing extra memory divergence. We consider (1) co-issuing instructions from different divergent paths of the same warp and (2) co-issuing instructions from different warps. To support (1), we introduce a novel thread reconvergence technique that ensures threads are run back in lockstep at control-flow reconvergence points without hindering their ability to run branches in parallel. We propose a lane shuffling technique to allow solution (2) to benefit from inter-warp correlations in divergence patterns. The combination of all these techniques improves performance by 23% on a set of regular GPGPU applications and by 40% on irregular applications, while maintaining the same instruction-fetch and processing-unit resource requirements as the contemporary Fermi GPU architecture.

GPU-based fast Monte Carlo simulation for radiotherapy dose calculation

Monte Carlo (MC) simulation is commonly considered to be the most accurate dose calculation method in radiotherapy. However, its efficiency still requires improvement for many routine clinical applications. In this paper, we present our recent progress toward the development of a graphics processing unit (GPU)-based MC dose calculation package, gDPM v2.0. It utilizes the parallel computation ability of a GPU to achieve high efficiency, while maintaining the same particle transport physics as in the original dose planning method (DPM) code and hence the same level of simulation accuracy. In GPU computing, divergence of execution paths between threads can considerably reduce the efficiency. Since photons and electrons undergo different physics and hence attain different execution paths, we use a simulation scheme where photon transport and electron transport are separated to partially relieve the thread divergence issue. A high-performance random number generator and a hardware linear interpolation are also utilized. We have also developed various components to handle the fluence map and linac geometry, so that gDPM can be used to compute dose distributions for realistic IMRT or VMAT treatment plans. Our gDPM package is tested for its accuracy and efficiency in both phantoms and realistic patient cases. In all cases, the average relative uncertainties are less than 1%. A statistical t-test is performed and the dose difference between the CPU and the GPU results is not found to be statistically significant in over 96% of the high dose region and over 97% of the entire region. Speed-up factors of 69.1 ∼ 87.2 have been observed using an NVIDIA Tesla C2050 GPU card against a 2.27 GHz Intel Xeon CPU processor. For realistic IMRT and VMAT plans, MC dose calculation can be completed with less than 1% standard deviation in 36.1 ∼ 39.6 s using gDPM.

TU‐E‐BRB‐09: Development of a GPU‐Based Monte Carlo Dose Calculation Package for Cancer Radiotherapy

Purpose: Monte Carlo (MC) simulation is commonly considered as the most accurate dose calculation method in radiotherapy. However, its efficiency still requires improvement for many routine clinical applications. The goal of this work is to develop a GPU‐based MC dose calculation package, gDPM. It utilizes the parallel computation ability of a GPU to achieve high efficiency, while maintaining same particle transport physics as in the original DPM code and hence simulation accuracy.Methods: The gDPM code is implemented on GPU architecture under CUDA platform. In GPU computing, divergence of execution paths between threads can considerably reduce the efficiency. Since photons and electrons undergo different physics and hence attain different execution path, we design a simulation scheme where photon transport and electron transport are separated to partially relieve the thread divergence issue. High performance random number generator and hardware linear interpolation are also utilized. We have also developed various components to account for fluence map and linac geometry, so that gDPM can be used to compute dose for real IMRT or VMAT treatment plans. The gDPM is tested for its accuracy and efficiency in both phantoms and realistic patient cases. Results: In all cases, the average relative uncertainties are less than 1%. Dose difference between CPU and GPU results is less than 3% of maximum dose in over 98% high dose region. A t‐test is also performed and the dose difference is not statistically significant in over 98% high dose region. Speed up factors of 57.3∼75.5 have been observed using an NVIDIA Tesla C2050 GPU card against a 2.27GHz Intel Xeon CPU processor. Dose calculation for a real IMRT head‐and‐neck plan and a VMAT prostate plan can be finished in 48sec and 46sec, respectively. Conclusions: The gDPM package has achieved high computation efficiency on GPU without losing accuracy.This work is supported in part by the University of California Lab Fees Research Program. GPU cards are partly provided by NVIDIA.