Scalable Parallel Random Number Generators Research Articles

GASPRNG: GPU accelerated scalable parallel random number generator library

Graphics processors represent a promising technology for accelerating computational science applications. Many computational science applications require fast and scalable random number generation with good statistical properties, so they use the Scalable Parallel Random Number Generators library (SPRNG). We present the GPU Accelerated SPRNG library (GASPRNG) to accelerate SPRNG in GPU-based high performance computing systems. GASPRNG includes code for a host CPU and CUDA code for execution on NVIDIA graphics processing units (GPUs) along with a programming interface to support various usage models for pseudorandom numbers and computational science applications executing on the CPU, GPU, or both. This paper describes the implementation approach used to produce high performance and also describes how to use the programming interface. The programming interface allows a user to be able to use GASPRNG the same way as SPRNG on traditional serial or parallel computers as well as to develop tightly coupled programs executing primarily on the GPU. We also describe how to install GASPRNG and use it. To help illustrate linking with GASPRNG, various demonstration codes are included for the different usage models. GASPRNG on a single GPU shows up to 280x speedup over SPRNG on a single CPU core and is able to scale for larger systems in the same manner as SPRNG. Because GASPRNG generates identical streams of pseudorandom numbers as SPRNG, users can be confident about the quality of GASPRNG for scalable computational science applications. Program summaryProgram title: GASPRNGCatalogue identifier: AEOI_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEOI_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: UTK license.No. of lines in distributed program, including test data, etc.: 167900No. of bytes in distributed program, including test data, etc.: 1422058Distribution format: tar.gzProgramming language: C and CUDA.Computer: Any PC or workstation with NVIDIA GPU (Tested on Fermi GTX480, Tesla C1060, Tesla M2070).Operating system: Linux with CUDA version 4.0 or later. Should also run on MacOS, Windows, or UNIX.Has the code been vectorized or parallelized?: Yes. Parallelized using MPI directives.RAM: 512 MB∼ 732 MB (main memory on host CPU, depending on the data type of random numbers.) / 512 MB (GPU global memory)Classification: 4.13, 6.5.Nature of problem:Many computational science applications are able to consume large numbers of random numbers. For example, Monte Carlo simulations are able to consume limitless random numbers for the computation as long as resources for the computing are supported. Moreover, parallel computational science applications require independent streams of random numbers to attain statistically significant results. The SPRNG library provides this capability, but at a significant computational cost. The GASPRNG library presented here accelerates the generators of independent streams of random numbers using graphical processing units (GPUs).Solution method:Multiple copies of random number generators in GPUs allow a computational science application to consume large numbers of random numbers from independent, parallel streams. GASPRNG is a random number generators library to allow a computational science application to employ multiple copies of random number generators to boost performance. Users can interface GASPRNG with software code executing on microprocessors and/or GPUs.Running time:The tests provided take a few minutes to run.

Scalable parallel physical random number generator based on a superluminescent LED

We describe an optoelectronic system for simultaneously generating parallel, independent streams of random bits using spectrally separated noise signals obtained from a single optical source. Using a pair of nonoverlapping spectral filters and a fiber-coupled superluminescent LED (SLED), we produced two independent 10 Gb/s random bit streams, for a cumulative generation rate of 20 Gb/s. The system relies principally on chip-based optoelectronic components that could be integrated in a compact, economical package.

Implementation of Hardware-Accelerated Scalable Parallel Random Number Generators

The Scalable Parallel Random Number Generators (SPRNGs) library is widely used in computational science applications such as Monte Carlo simulations since SPRNG supports fast, parallel, and scalable random number generation with good statistical properties. In order to accelerate SPRNG, we develop a Hardware-Accelerated version of SPRNG (HASPRNG) on the Xilinx XC2VP50 Field Programmable Gate Arrays (FPGAs) in the Cray XD1 that produces identical results. HASPRNG includes the reconfigurable logic for FPGAs along with a programming interface which performs integer random number generation. To demonstrate HASPRNG for Reconfigurable Computing (RC) applications, we also develop a Monte Carlo π-estimator for the Cray XD1. The RC Monte Carlo π-estimator shows a 19.1× speedup over the 2.2 GHz AMD Opteron processor in the Cray XD1. In this paper we describe the FPGA implementation for HASPRNG and a π-estimator example application exploiting the fine-grained parallelism and mathematical properties of the SPRNG algorithm.

HASPRNG: Hardware Accelerated Scalable Parallel Random Number Generators

The Scalable Parallel Random Number Generators library (SPRNG) supports fast and scalable random number generation with good statistical properties for parallel computational science applications. In order to accelerate SPRNG in high performance reconfigurable computing systems, we present the Hardware Accelerated SPRNG library (HASPRNG). Ported to the Xilinx University Program (XUP) and Cray XD1 reconfigurable computing platforms, HASPRNG includes the reconfigurable logic for Field Programmable Gate Arrays (FPGAs) along with a programming interface which performs integer random number generation that produces identical results with SPRNG. This paper describes the reconfigurable logic of HASPRNG exploiting the mathematical properties and data parallelism residing in the SPRNG algorithms to produce high performance and also describes how to use the programming interface to minimize the communication overhead between FPGAs and microprocessors. The programming interface allows a user to be able to use HASPRNG the same way as SPRNG 2.0 on platforms such as the Cray XD1. We also describe how to install HASPRNG and use it. For HASPRNG usage we discuss a FPGA π-estimator for a High Performance Reconfigurable Computer (HPRC) sample application and compare to a software π-estimator. HASPRNG shows 1.7x speedup over SPRNG on the Cray XD1 and is able to obtain substantial speedup for a HPRC application. Program summary Program title: HASPRNG Catalogue identifier: AEER_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEER_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 594 928 No. of bytes in distributed program, including test data, etc.: 6 509 724 Distribution format: tar.gz Programming language: VHDL (XUP and Cray XD1), C++ (XUP), C (Cray XD1) Computer: PowerPC 405 (XUP) / AMD 2.2 GHz Opteron processor (Cray XD1) Operating system: Linux File size: 15 MB (XUP) / 22 MB (Cray XD1) Classification: 4.13 Nature of problem: Many computational science applications are able to consume large numbers of random numbers. For example, Monte Carlo simulations such as π-estimation are able to consume limitless random numbers forthe computation as long as hardware resources for the computing are supported. Moreover, parallel computational science applications require independent streams of random numbers to attain statistically significant results. The SPRNG library provides this capability, but at a significant computational cost. The library presented here accelerates the generators of independent streams of random numbers. Solution method: Multiple copies of random number generators in FPGAs allow a computational science application to consume large numbers of random numbers from independent, parallel streams. HASPRNG is a random number generators library to allow a computational science application to employ the multiple copies of random number generators to boost performance. Users can interface HASPRNG with software code executing on microprocessors and/or with hardware applications executing on FPGAs.

A decentralized parallel implementation for parallel tempering algorithm

Parallel tempering (PT), also known as replica exchange, is a powerful Markov Chain Monte Carlo sampling approach, which aims at reducing the relaxation time in simulations of physical systems. In this paper, we present a novel decentralized parallel implementation of PT using the message passing interface (MPI) and the scalable parallel random number generators (SPRNG) library. By taking advantage of the characteristics of pseudorandom number generators, this implementation eliminates global synchronization and reduces the overhead caused by interprocessor communication in replica exchange in PT. Moreover, our proposed non-blocking replica exchange reduces communication overhead in pair-wise process replica exchanges by allowing the process reaching the replica exchange point to leap-ahead while waiting for the other one to reach the common replica exchange point. Also, temperature exchange instead of conformation replica exchange is proposed to reduce communication and achieve load balancing in the participating processors in the PT computation. All these enable one to efficiently apply PT to large-scale massively parallel systems. The efficiency of this parallel PT implementation is demonstrated in the context of minimizing various benchmark functions with complicated landscapes as objective functions. Our computational results and analysis have shown that the decentralized PT is scalable, reproducible, load-balanced, and yields insignificant communication overhead.

Parallel linear congruential generators with Sophie–Germain moduli

Monte Carlo simulations are thought to be very easy to parallelize; however, the quality of these parallel Monte Carlo computations depends greatly on the quality of the parallel random number generators used. Linear congruential generators (LCGs), the most common number-theoretic pseudorandom number generators, with both power-of-two and prime moduli are used in many popular implementations of pseudorandom number generators. Recently, one of the authors of this paper [M. Mascagni, Parallel linear congruential generators with prime moduli, Parallel Comput. 24 (1998) 923–936] developed an explicit parameterization of prime modulus LCGs for use in parallel computations. This approach was based on an explicit enumeration of all the primitive roots modulo the prime modulus for use as unique multipliers in each parallel LCG. In that paper, only Mersenne prime moduli were considered because of the existence of a fast modular multiplication algorithm for primes close to powers-of-two. In the current paper, we investigate the nature of the trade-off implicitly made in the choice of Mersenne primes by comparing them to parameterized Sophie–Germain prime modulus LCGs. While the choice of Mersenne primes trades off initialization time for generation time, the choice of Sophie–Germain primes not only largely reduces initialization time but also provides competitive generation time when an appropriately chosen Sophie–Germain primes are used. The resulting Sophie–Germain prime modulus LCGs have been tested, and incorporated into the Scalable Parallel Random Number Generators SPRNG library [SPRNG. Scalable parallel random number generators, http://sprng.fsu.edu], a widely used random number generation suite for parallel, distributed, and grid-based Monte Carlo computations [M. Mascagni, A. Srinivasan, Computational infrastructure for parallel, distributed, and grid-based Monte Carlo computations, Lect. Notes Comput. Sci. 2907 (2004) 39–52].

Parameterizing parallel multiplicative lagged-Fibonacci generators

Monte Carlo computations are commonly considered to be naturally parallel. However, one needs to exercise care in parallelizing the underlying pseudo random number generator (PRNG) to avoid correlations within, and between, random number streams. PRNGs are normally parallelized using one of the following two paradigms: (i) cycle division and (ii) parameterization. Most of the popular PRNGs based on linear recurrences, such as linear congruential generators (LCGs), generalized feedback shift- register generators (GFSRs), and additive lagged- Fibonacci generators (ALFGs) have been parallelized using both these paradigms. While the (nonlinear) multiplicative lagged- Fibonacci generators (MLFG) is considered superior to the ALFG in quality, it had so far been parallelized only via the cycle division paradigm, which has its limitations. In this paper, we describe parallelization of the MLFG using parameterization, and discuss its implementation in the Scalable Parallel Random Number Generators ( SPRNG) [ACM Trans. Math. Software 26 (2000) 436] parallel pseudorandom number generation software. We also present empirical results demonstrating the quality, and quantitatively compare the parallel ALFGs and MLFGs quality.

Testing parallel random number generators

Monte Carlo computations are considered easy to parallelize. However, the results can be adversely affected by defects in the parallel pseudorandom number generator used. A parallel pseudorandom number generator must be tested for two types of correlations––(i) intra-stream correlation, as for any sequential generator, and (ii) inter-stream correlation for correlations between random number streams on different processes. Since bounds on these correlations are difficult to prove mathematically, large and thorough empirical tests are necessary. Many of the popular pseudorandom number generators in use today were tested when computational power was much lower, and hence they were evaluated with much smaller test sizes. This paper describes several tests of pseudorandom number generators, both statistical and application-based. We show defects in several popular generators. We describe the implementation of these tests in the SPRNG [ACM Trans. Math. Software 26 (2000) 436; SPRNG––scalable parallel random number generators. SPRNG 1.0–– http://www.ncsa.uiuc.edu/Apps/SPRNG ; SPRNG 2.0–– http://sprng.cs.fsu.edu ] test suite and also present results for the tests conducted on the SPRNG generators. These generators have passed some of the largest empirical random number tests.

A parallel Monte Carlo code for planar and SPECT imaging: implementation, verification and applications in 131I SPECT

This paper reports the implementation of the SIMIND Monte Carlo code on an IBM SP2 distributed memory parallel computer. Basic aspects of running Monte Carlo particle transport calculations on parallel architectures are described. Our parallelization is based on equally partitioning photons among the processors and uses the Message Passing Interface (MPI) library for interprocessor communication and the Scalable Parallel Random Number Generator (SPRNG) to generate uncorrelated random number streams. These parallelization techniques are also applicable to other distributed memory architectures. A linear increase in computing speed with the number of processors is demonstrated for up to 32 processors. This speed-up is especially significant in Single Photon Emission Computed Tomography (SPECT) simulations involving higher energy photon emitters, where explicit modeling of the phantom and collimator is required. For 131I, the accuracy of the parallel code is demonstrated by comparing simulated and experimental SPECT images from a heart/thorax phantom. Clinically realistic SPECT simulations using the voxel-man phantom are carried out to assess scatter and attenuation correction.

An efficient backward Monte Carlo estimator for solving of a quantum-kinetic equation with memory kernel

An efficient backward Monte Carlo (MC) estimator and a corresponding algorithm for solving a quantum-kinetic equation describing an ultrafast semiconductor carrier transport is proposed and studied. In order to obtain the electron energy distribution for long evolution times, variance reduction techniques are applied. The balancing of errors (both systematic and stochastic) and computational cost are investigated. The presented algorithm is implemented using the scalable parallel random number generator (SPRNG) and one by P. L’Ecuyer based on a combination of two linear congruential sequences. Numerical results for long and short evolution times are obtained. They show that the SPRNG is preferable to that by P. L’Ecuyer.

Corrigendum: Algorithm 806

In this article we present background, rationale, and a description of the Scalable Parallel Random Number Generators (SPRNG) library. We begin by presenting some methods for parallel pseudorandom number generation. We will focus on methods based on parameterization, meaning that we will not consider splitting methods such as the leap-frog or blocking methods. We describe, in detail, parameterized versions of the following pseudorandom number generators: (1) linear congruential generators, (ii) shift-register generators, and (iii) lagged-Fibonacci generators. We briefly describe the methods, detail some advantages and disadvantages of each method, and recount results from number theory that impact our understanding of their quality of parallel applications. SPRNG was designed around the uniform implementation of different families of parameterized random number generators. We then present a short description of SPRNG. The description contained within this document is meant only to outline the rationale behind and the capabilities of SPRNG. Much more information, including examples and detailed documentation aimed at helping users with putting and using SPRNG on scalable systems is available at http://sprng.cs.fsu.edu. In this description of SPRNG we discuss the random-number generator library as well as the suite of tests of randomness that is an integral part of SPRNG. Random-number tools for parallel Monte Carlo applications must be subjected to classical as well as new types of empirical tests of randomness to eliminate generators that show defects when used in scalable environments.

Algorithm 806: SPRNG

In this article we present background, rationale, and a description of the Scalable Parallel Random Number Generators (SPRNG) library. We begin by presenting some methods for parallel pseudorandom number generation. We will focus on methods based on parameterization, meaning that we will not consider splitting methods such as the leap-frog or blocking methods. We describe, in detail, parameterized versions of the following pseudorandom number generators: (i) linear congruential generators, (ii) shift-register generators, and (iii) lagged-Fibonacci generators. We briefly describe the methods, detail some advantages and disadvantages of each method, and recount results from number theory that impact our understanding of their quality in parallel applications. SPRNG was designed around the uniform implementation of different families of parameterized random number generators. We then present a short description of SPRNG. The description contained within this document is meant only to outline the rationale behind and the capabilities of SPRNG. Much more information, including examples and detailed documentation aimed at helping users with putting and using SPRNG on scalable systems is available at htt;//sprng.sc.fsu.edu. In this description of SPRNG we discuss the random-number generator library as well as the suite of tests of randomness that is an integral part of SPRNG. Random-number tools for parallel Monte Carlo applications must be subjected to classical as well as new types of empirical tests of randomness to eliminate generators that show defects when used in scalable envionments.