Parallel Pseudorandom Number Generation Research Articles

A Deterministic Portable Parallel Pseudo-Random Number Generator for Pattern-Based Programming of Heterogeneous Parallel Systems

SkePU is a pattern-based high-level programming model for transparent program execution on heterogeneous parallel computing systems. A key feature of SkePU is that, in general, the selection of the execution platform for a skeleton-based function call need not be determined statically. On single-node systems, SkePU can select among CPU, multithreaded CPU, single or multi-GPU execution. Many scientific applications use pseudo-random number generators (PRNGs) as part of the computation. In the interest of correctness and debugging, deterministic parallel execution is a desirable property, which however requires a deterministically parallelized pseudo-random number generator. We present the API and implementation of a deterministic, portable parallel PRNG extension to SkePU that is scalable by design and exhibits the same behavior regardless where and with how many resources it is executed. We evaluate it with four probabilistic applications and show that the PRNG enables scalability on both multi-core CPU and GPU resources, and hence supports the universal portability of SkePU code even in the presence of PRNG calls, while source code complexity is reduced.

Robust parallel hybrid artificial bee colony algorithms for the multi-dimensional numerical optimization

This study proposes a set of new robust parallel hybrid metaheuristic algorithms based on artificial bee colony (ABC) and teaching learning-based optimization (TLBO) for the multi-dimensional numerical problems. The best practices of ABC and TLBO are implemented to provide robust algorithms on a distributed memory computation environment using MPI libraries. Island parallel versions of the proposed hybrid algorithm are observed to obtain much better results than those of sequential versions. Parallel pseudorandom number generators are used to provide diverse solution candidates to prevent stagnation into local optima. The performances of the proposed hybrid algorithms are compared with eight different metaheuristics algorithms of particle swarm optimization, differential evolution variants, ABC variants and evolutionary algorithm. The empirical results show that the new hybrid parallel algorithms are scalable and the best performing algorithms when compared to the state-of-the-art metaheuristics.

The development of GPU-based parallel PRNG for Monte Carlo applications in CUDA Fortran

The implementation of Monte Carlo simulation on the CUDA Fortran requires a fast random number generation with good statistical properties on GPU. In this study, a GPU-based parallel pseudo random number generator (GPPRNG) have been proposed to use in high performance computing systems. According to the type of GPU memory usage, GPU scheme is divided into two work modes including GLOBAL_MODE and SHARED_MODE. To generate parallel random numbers based on the independent sequence method, the combination of middle-square method and chaotic map along with the Xorshift PRNG have been employed. Implementation of our developed PPRNG on a single GPU showed a speedup of 150x and 470x (with respect to the speed of PRNG on a single CPU core) for GLOBAL_MODE and SHARED_MODE, respectively. To evaluate the accuracy of our developed GPPRNG, its performance was compared to that of some other commercially available PPRNGs such as MATLAB, FORTRAN and Miller-Park algorithm through employing the specific standard tests. The results of this comparison showed that the developed GPPRNG in this study can be used as a fast and accurate tool for computational science applications.

Analysis of Elementary Cellular Automata Boundary Conditions

We present the findings of analysis of elementary cellular automata (ECA) boundary conditions. Fixed and variable boundaries are attempted. The outputs of linear feedback shift registers (LFSRs) act as continuous inputs to the two boundaries of a one-dimensional (1-D) Elementary Cellular Automata (ECA) are analyzed and compared. The results show superior randomness features and the output string has passed the Diehard statistical battery of tests. The design has strong correlation immunity and it is inherently amenable for VLSI implementation. Therefore it can be considered to be a good and viable candidate for parallel pseudo random number generation.

Parallel pseudo-random number generators: A derivative pricing perspective with the Heston stochastic volatility model

Accuracy and precision of parallel Monte Carlo (MC) simulations may be impaired by the presence of intra-thread and inter-thread correlations depending on the parallel pseudo-random number generators (PPRNGs) used. While necessary, statistical tests alone are insufficient to ensure the absence of these correlations that can manifest as bias and variance to a extent in different applications. Therefore, application-based tests designed to mimic real-life MC scenarios may uncover them in the intended applications. The results of an application-based test simulating the Heston stochastic volatility model, a widely used pricing framework, is reported in order to compare the accuracy and precision profiles among four popular libraries of scalable pseudo-random number generators implementing sequence division ( trng and RngSteam ), parameterization ( sprng ) and counter-based ( Random123 ) strategies. All pseudo-random number generators assessed demonstrate similar standard-error of mean profiles. However, the bias profiles are more varied albeit comparable. PPRNGs demonstrating the smallest bias profiles in absolute and relative terms are yarn4 from TRNG , mlfg from SPRNG , as well as Threefry2x64 from Random123 for truncated Euler scheme, and mrg5s from TRNG and lfg from SPRNG for the quadratic exponent scheme. It is recommended that, when selecting a PPRNG for parallel MC simulation from a set of competing PPRNGs with comparable bias and standard error of mean profiles in absolute terms, the PPRNG associated with the smallest parallel-sequential bias difference should be used as it reflects the smallest intra-thread correlation introduced by parallelization.

Parallel random number generators in Monte Carlo derivative pricing: An application-based test

Parallel pseudorandom number generators (PPRNG) that satisfy classical statistical tests may still demonstrate intra-stream and inter-stream correlations in real life applications. In order to investigate the suitability of a PPRNG for use in Monte Carlo pricing of financial derivatives, an application-based test is proposed to evaluate the bias and the standard error of the mean (SE) associated with the PPRNG as a gauge of intra-stream and inter-stream correlations respectively. This test involves estimating the price of a vanilla European call option via Monte Carlo simulation, where the asset price at maturity is estimated by propagating the Black–Scholes stochastic differential equation via the Euler–Maruyama discretization scheme. The mean and SE profiles of the numerical results based on three PPRNG libraries (RngStream, TRNG and SPRNG) that implement parallel random numbers via sequence splitting strategies (RngStream and TRNG) and parameterization strategy (SPRNG) are compared. In terms of the bias and SE profiles, the best performing PPRNG constructed using the sequence splitting strategy is comparable to that constructed using parameterization, both use multiple recursive generators in their kernel.

DEMCMC-GPU: An Efficient Multi-Objective Optimization Method with GPU Acceleration on the Fermi Architecture

In this paper, we present an efficient method implemented on Graphics Processing Unit (GPU), DEMCMC-GPU, for multi-objective continuous optimization problems. The DEMCMC-GPU kernel is the DEMCMC algorithm, which combines the attractive features of Differential Evolution (DE) and Markov Chain Monte Carlo (MCMC) to evolve a population of Markov chains toward a diversified set of solutions at the Pareto optimal front in the multi-objective search space. With parallel evolution of a population of Markov chains, the DEMCMC algorithm is a natural fit for the GPU architecture. The implementation of DEMCMC-GPU on the pre-Fermi architecture can lead to a ~25 speedup on a set of multi-objective benchmark function problems, compare to the CPU-only implementation of DEMCMC. By taking advantage of new cache mechanism in the emerging NVIDIA Fermi GPU architecture, efficient sorting algorithm on GPU, and efficient parallel pseudorandom number generators, the speedup of DEMCMC-GPU can be aggressively improved to ~100.

Quasi-Monte Carlo integration on the grid for sensitivity studies

In this paper we present error and performance analysis of quasi-Monte Carlo algorithms for solving multidimensional integrals (up to 100 dimensions) on the grid using MPI. We take into account the fact that the Grid is a potentially heterogeneous computing environment, where the user does not know the specifics of the target architecture. Therefore parallel algorithms should be able to adapt to this heterogeneity, providing automated load-balancing. Monte Carlo algorithms can be tailored to such environments, provided parallel pseudorandom number generators are available. The use of quasi-Monte Carlo algorithms poses more difficulties. In both cases the efficient implementation of the algorithms depends on the functionality of the corresponding packages for generating pseudorandom or quasirandom numbers. We propose efficient parallel implementation of the Sobol sequence for a grid environment and we demonstrate numerical experiments on a heterogeneous grid. To achieve high parallel efficiency we use a newly developed special grid service called Job Track Service which provides efficient management of available computing resources through reservations.

Parallelization of a Monte Carlo particle transport simulation code

We have developed a high performance version of the Monte Carlo particle transport simulation code MC4. The original application code, developed in Visual Basic for Applications (VBA) for Microsoft Excel, was first rewritten in the C programming language for improving code portability. Several pseudo-random number generators have been also integrated and studied. The new MC4 version was then parallelized for shared and distributed-memory multiprocessor systems using the Message Passing Interface. Two parallel pseudo-random number generator libraries (SPRNG and DCMT) have been seamlessly integrated. The performance speedup of parallel MC4 has been studied on a variety of parallel computing architectures including an Intel Xeon server with 4 dual-core processors, a Sun cluster consisting of 16 nodes of 2 dual-core AMD Opteron processors and a 200 dual-processor HP cluster. For large problem size, which is limited only by the physical memory of the multiprocessor server, the speedup results are almost linear on all systems. We have validated the parallel implementation against the serial VBA and C implementations using the same random number generator. Our experimental results on the transport and energy loss of electrons in a water medium show that the serial and parallel codes are equivalent in accuracy. The present improvements allow for studying of higher particle energies with the use of more accurate physical models, and improve statistics as more particles tracks can be simulated in low response time.

Parallel Quasirandom Applications on Heterogeneous Grid

In this paper we present error and performance analysis of quasi-Monte Carlo algorithms for solving multidimensional integrals (up to 100 dimensions) on the grid using MPI. We take into account the fact that the Grid is a potentially heterogeneous computing environment, where the user does not know the specifics of the target architecture. Therefore parallel algorithms should be able to adapt to this heterogeneity, providing automated load-balancing. Monte Carlo algorithms can be tailored to such environments, provided parallel pseudorandom number generators are available. The use of quasi-Monte Carlo algorithms poses more difficulties. In both cases the efficient implementation of the algorithms depends on the functionality of the corresponding packages for generating pseudorandom or quasirandom numbers. We propose efficient parallel implementation of the Sobol sequence for a grid environment and we demonstrate numerical experiments on a heterogeneous grid. To achieve high parallel efficiency we use a newly developed special grid service called Job Track Service which provides efficient management of available computing resources through reservations.

Parameterizing parallel multiplicative lagged-Fibonacci generators

Monte Carlo computations are commonly considered to be naturally parallel. However, one needs to exercise care in parallelizing the underlying pseudo random number generator (PRNG) to avoid correlations within, and between, random number streams. PRNGs are normally parallelized using one of the following two paradigms: (i) cycle division and (ii) parameterization. Most of the popular PRNGs based on linear recurrences, such as linear congruential generators (LCGs), generalized feedback shift- register generators (GFSRs), and additive lagged- Fibonacci generators (ALFGs) have been parallelized using both these paradigms. While the (nonlinear) multiplicative lagged- Fibonacci generators (MLFG) is considered superior to the ALFG in quality, it had so far been parallelized only via the cycle division paradigm, which has its limitations. In this paper, we describe parallelization of the MLFG using parameterization, and discuss its implementation in the Scalable Parallel Random Number Generators ( SPRNG) [ACM Trans. Math. Software 26 (2000) 436] parallel pseudorandom number generation software. We also present empirical results demonstrating the quality, and quantitatively compare the parallel ALFGs and MLFGs quality.

Testing parallel random number generators

Monte Carlo computations are considered easy to parallelize. However, the results can be adversely affected by defects in the parallel pseudorandom number generator used. A parallel pseudorandom number generator must be tested for two types of correlations––(i) intra-stream correlation, as for any sequential generator, and (ii) inter-stream correlation for correlations between random number streams on different processes. Since bounds on these correlations are difficult to prove mathematically, large and thorough empirical tests are necessary. Many of the popular pseudorandom number generators in use today were tested when computational power was much lower, and hence they were evaluated with much smaller test sizes. This paper describes several tests of pseudorandom number generators, both statistical and application-based. We show defects in several popular generators. We describe the implementation of these tests in the SPRNG [ACM Trans. Math. Software 26 (2000) 436; SPRNG––scalable parallel random number generators. SPRNG 1.0–– http://www.ncsa.uiuc.edu/Apps/SPRNG ; SPRNG 2.0–– http://sprng.cs.fsu.edu ] test suite and also present results for the tests conducted on the SPRNG generators. These generators have passed some of the largest empirical random number tests.

The PLFG parallel pseudo-random number generator

Parallel computing has been touted as the pinnacle of high performance digital computing by many. However, many problems remain intractable using deterministic algorithms. Randomized algorithms which are, in some cases, less efficient than their deterministic counterpart for smaller problem sizes, can overturn the intractability of various large-scale problems. These algorithms, however, require a source of randomness. Pseudo-random number generators were created for many of these purposes. When performing computations on parallel machines, an additional criterion for randomized algorithms to be worthwhile is the availability of a parallel pseudo-random number generator. This paper presents an efficient algorithm for parallel pseudo-random number generation.

A new empirical test for parallel pseudo-random number generators

Recently, Percus derived probabilities and distributions for parallel, i.i.d. random sequences of integers. This was accomplished by considering s given bit locations in each random variable (represented as a predetermined number of bits) in each sequence. These s bits were used to create a new binary sequence whose expected behavior can be analyzed. Based upon Percus work, an empirical test for parallel pseudo-random number generators has been devised. For each generator, parallel sequences of various lengths are considered and analyzed as proposed by Percus and the results are statistically compared to the expected behavior for truly random sequences. A variety of parallel pseudo-random number generators from the literature are studied and the usefulness of the new empirical test is discussed.

Corrigendum: Algorithm 806

In this article we present background, rationale, and a description of the Scalable Parallel Random Number Generators (SPRNG) library. We begin by presenting some methods for parallel pseudorandom number generation. We will focus on methods based on parameterization, meaning that we will not consider splitting methods such as the leap-frog or blocking methods. We describe, in detail, parameterized versions of the following pseudorandom number generators: (1) linear congruential generators, (ii) shift-register generators, and (iii) lagged-Fibonacci generators. We briefly describe the methods, detail some advantages and disadvantages of each method, and recount results from number theory that impact our understanding of their quality of parallel applications. SPRNG was designed around the uniform implementation of different families of parameterized random number generators. We then present a short description of SPRNG. The description contained within this document is meant only to outline the rationale behind and the capabilities of SPRNG. Much more information, including examples and detailed documentation aimed at helping users with putting and using SPRNG on scalable systems is available at http://sprng.cs.fsu.edu. In this description of SPRNG we discuss the random-number generator library as well as the suite of tests of randomness that is an integral part of SPRNG. Random-number tools for parallel Monte Carlo applications must be subjected to classical as well as new types of empirical tests of randomness to eliminate generators that show defects when used in scalable environments.

Algorithm 806: SPRNG

In this article we present background, rationale, and a description of the Scalable Parallel Random Number Generators (SPRNG) library. We begin by presenting some methods for parallel pseudorandom number generation. We will focus on methods based on parameterization, meaning that we will not consider splitting methods such as the leap-frog or blocking methods. We describe, in detail, parameterized versions of the following pseudorandom number generators: (i) linear congruential generators, (ii) shift-register generators, and (iii) lagged-Fibonacci generators. We briefly describe the methods, detail some advantages and disadvantages of each method, and recount results from number theory that impact our understanding of their quality in parallel applications. SPRNG was designed around the uniform implementation of different families of parameterized random number generators. We then present a short description of SPRNG. The description contained within this document is meant only to outline the rationale behind and the capabilities of SPRNG. Much more information, including examples and detailed documentation aimed at helping users with putting and using SPRNG on scalable systems is available at htt;//sprng.sc.fsu.edu. In this description of SPRNG we discuss the random-number generator library as well as the suite of tests of randomness that is an integral part of SPRNG. Random-number tools for parallel Monte Carlo applications must be subjected to classical as well as new types of empirical tests of randomness to eliminate generators that show defects when used in scalable envionments.

Parallel pseudorandom number generation in GaAs cellular automata for high speed circuit testing

To facilitate test vector generation for high-speed circuits, we present the design and circuit simulation of parallel pseudorandom number generators in GaAs technology. These PRNGs are based on hybrid cellular automata (CA) in which mixtures of local rules are employed in one dimensional arrays, with minimal delay due to having only local wiring between neighboring cells. HSPICE simulations of these circuits demonstrate that they operate at a clock frequency above 1 GHz. Delay simulations indicate that GaAs PRNGs based upon linear feedback shift registers, in contrast with hybrid CAs, exhibit a degradation in clock frequency due to the effects of global interconnects, and that this degradation increases with the register length.