Independent Random Number Streams Research Articles

Employing AVX vectorization to improve the performance of random number generators

By the example of the RNGAVXLIB random number generator library, this paper considers some approaches to employing AVX vectorization for calculation speedup. The RNGAVXLIB library contains AVX implementations of modern generators and the routines allowing one to initialize up to 1019 independent random number streams. The AVX implementations yield exactly the same pseudorandom sequences as the original algorithms do, while being up to 40 times faster than the ANSI C implementations.

Random numbers for parallel computers: Requirements and methods, with emphasis on GPUs

We examine the requirements and the available methods and software to provide (or imitate) uniform random numbers in parallel computing environments. In this context, for the great majority of applications, independent streams of random numbers are required, each being computed on a single processing element at a time. Sometimes, thousands or even millions of such streams are needed. We explain how they can be produced and managed. We devote particular attention to multiple streams for GPU devices.

RNGAVXLIB: Program library for random number generation, AVX realization

Abstract The library RNGSSELIB for random number generators (RNGs) based upon the SSE2 command set is presented. The library contains realization of a number of modern and most reliable generators. Usage of SSE2 command set allows to substantially improve performance of the generators. Three new RNG realizations are also constructed. We present detailed analysis of the speed depending on compiler usage and associated optimization level, as well as results of extensive statistical testing for all generators using available test packages. Fast SSE implementations produce exactly the same output sequence as the original algorithms. Program summary Program title: RNGSSELIB Catalogue identifier: AEIT_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEIT_v1_0.html Program obtainable from: CPC Program Library, Queenʼs University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 4177 No. of bytes in distributed program, including test data, etc.: 21 228 Distribution format: tar.gz Programming language: C. Computer: PC. Operating system: UNIX, Windows. RAM: 1 Mbytes Classification: 4.13. Nature of problem: Any calculation requiring uniform pseudorandom number generator, in particular, Monte Carlo calculations. Solution method: The library contains realization of a number of modern and reliable generators: mt19937 , mrg32k3a and lfsr113 . Also new realizations for the method based on parallel evolution of an ensemble of dynamical systems are constructed: GM19 , GM31 and GM61 . The library contains both usual realizations and realizations based on SSE command set. Usage of SSE commands allows the performance of all generators to be substantially improved. Restrictions: For SSE realizations of the generators, Intel or AMD CPU supporting SSE2 command set is required. In order to use the realization lfsr113sse , CPU must support SSE4 command set. Running time: Running time is of the order of 20 sec for generating 109 pseudorandom numbers with a PC based on Intel Core i7-940 CPU. Running time is analysed in detail in Section 5 of the paper.

Using RngStreams for parallel random number generation in C++ and R

The RngStreams software package provides one viable solution to the problem of creating independent random number streams for simulations in parallel processing environments. Techniques are presented for effectively using RngStreams with C++ programs that are parallelized via OpenMP or MPI. Ways to access the backbone generator from RngStreams in R through the parallel and rstream packages are also described. The ideas in the paper are illustrated with both a simple running example and a Monte Carlo integration application.

PRAND: GPU accelerated parallel random number generation library: Using most reliable algorithms and applying parallelism of modern GPUs and CPUs

The library PRAND for pseudorandom number generation for modern CPUs and GPUs is presented. It contains both single-threaded and multi-threaded realizations of a number of modern and most reliable generators recently proposed and studied in Barash (2011), Matsumoto and Tishimura (1998), L’Ecuyer (1999,1999), Barash and Shchur (2006) and the efficient SIMD realizations proposed in Barash and Shchur (2011). One of the useful features for using PRAND in parallel simulations is the ability to initialize up to 1019 independent streams. Using massive parallelism of modern GPUs and SIMD parallelism of modern CPUs substantially improves performance of the generators. Program summaryProgram title: PRANDCatalogue identifier: AESB_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AESB_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 45979No. of bytes in distributed program, including test data, etc.: 23953564Distribution format: tar.gzProgramming language: Cuda C, Fortran.Computer: PC, workstation, laptop, or server with NVIDIA GPU (tested on Tesla X2070, Fermi C2050, GeForce GT540M) and with Intel or AMD processor.Operating system: Linux with CUDA version 5.0 or later. Should also run on MacOs, Windows, or UNIX.RAM: 4 MbytesClassification: 4.13.Nature of problem:Any calculation requiring uniform pseudorandom number generator, in particular, Monte Carlo calculations. Any calculation or simulation requiring uncorrelated parallel streams of uniform pseudorandom numbers.Solution method:The library contains realization of a number of modern and reliable generators: MT19937, MRG32K3A and LFSR113. Also new realizations of the method based on parallel evolution of an ensemble of transformations of two-dimensional torus are included in the library: GM19, GM29, GM31, GM61, GM55, GQ58.1, GQ58.3 and GQ58.4. The library contains: single-threaded and multi-threaded realizations for GPU, single-threaded realizations for CPU, realizations for CPU based on SSE command set. Also, the library contains the abilities to jump ahead inside RNG sequence and to initialize independent random number streams with block splitting method for each of the RNGs.Restrictions:Nvidia Cuda Toolkit version 5.0 or later should be installed. To use GPU realizations, Nvidia GPU supporting CUDA and the corresponding Nvidia driver should be installed. For SSE realizations of the generators, Intel or AMD CPU supporting SSE2 command set is required. In order to use the SSE realization of LFSR113, CPU must support SSE4 command set.Additional comments:A version of this program, which only contains the realizations for CPUs, is held in the Library as Catalog Id., AEIT_v2_0 (RNGSSELIB). It does not require a GPU device or CUDA compiler.Running time:The tests and the examples included in the package take less or about one minute to run. Running time is analyzed in detail in Section 8 of the paper.

RNGSSELIB: Program library for random number generation. More generators, parallel streams of random numbers and Fortran compatibility

In this update, we present the new version of the random number generator (RNG) library RNGSSELIB, which, in particular, contains fast SSE realizations of a number of modern and most reliable generators \cite{RNGSSELIB1}. The new features are: i) Fortran compatibility and examples of using the library in Fortran; ii) new modern and reliable generators; iii) the abilities to jump ahead inside RNG sequence and to initialize up to $10^{19}$ independent random number streams with block splitting method.

GASPRNG: GPU accelerated scalable parallel random number generator library

Graphics processors represent a promising technology for accelerating computational science applications. Many computational science applications require fast and scalable random number generation with good statistical properties, so they use the Scalable Parallel Random Number Generators library (SPRNG). We present the GPU Accelerated SPRNG library (GASPRNG) to accelerate SPRNG in GPU-based high performance computing systems. GASPRNG includes code for a host CPU and CUDA code for execution on NVIDIA graphics processing units (GPUs) along with a programming interface to support various usage models for pseudorandom numbers and computational science applications executing on the CPU, GPU, or both. This paper describes the implementation approach used to produce high performance and also describes how to use the programming interface. The programming interface allows a user to be able to use GASPRNG the same way as SPRNG on traditional serial or parallel computers as well as to develop tightly coupled programs executing primarily on the GPU. We also describe how to install GASPRNG and use it. To help illustrate linking with GASPRNG, various demonstration codes are included for the different usage models. GASPRNG on a single GPU shows up to 280x speedup over SPRNG on a single CPU core and is able to scale for larger systems in the same manner as SPRNG. Because GASPRNG generates identical streams of pseudorandom numbers as SPRNG, users can be confident about the quality of GASPRNG for scalable computational science applications. Program summaryProgram title: GASPRNGCatalogue identifier: AEOI_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEOI_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: UTK license.No. of lines in distributed program, including test data, etc.: 167900No. of bytes in distributed program, including test data, etc.: 1422058Distribution format: tar.gzProgramming language: C and CUDA.Computer: Any PC or workstation with NVIDIA GPU (Tested on Fermi GTX480, Tesla C1060, Tesla M2070).Operating system: Linux with CUDA version 4.0 or later. Should also run on MacOS, Windows, or UNIX.Has the code been vectorized or parallelized?: Yes. Parallelized using MPI directives.RAM: 512 MB∼ 732 MB (main memory on host CPU, depending on the data type of random numbers.) / 512 MB (GPU global memory)Classification: 4.13, 6.5.Nature of problem:Many computational science applications are able to consume large numbers of random numbers. For example, Monte Carlo simulations are able to consume limitless random numbers for the computation as long as resources for the computing are supported. Moreover, parallel computational science applications require independent streams of random numbers to attain statistically significant results. The SPRNG library provides this capability, but at a significant computational cost. The GASPRNG library presented here accelerates the generators of independent streams of random numbers using graphical processing units (GPUs).Solution method:Multiple copies of random number generators in GPUs allow a computational science application to consume large numbers of random numbers from independent, parallel streams. GASPRNG is a random number generators library to allow a computational science application to employ multiple copies of random number generators to boost performance. Users can interface GASPRNG with software code executing on microprocessors and/or GPUs.Running time:The tests provided take a few minutes to run.

HASPRNG: Hardware Accelerated Scalable Parallel Random Number Generators

The Scalable Parallel Random Number Generators library (SPRNG) supports fast and scalable random number generation with good statistical properties for parallel computational science applications. In order to accelerate SPRNG in high performance reconfigurable computing systems, we present the Hardware Accelerated SPRNG library (HASPRNG). Ported to the Xilinx University Program (XUP) and Cray XD1 reconfigurable computing platforms, HASPRNG includes the reconfigurable logic for Field Programmable Gate Arrays (FPGAs) along with a programming interface which performs integer random number generation that produces identical results with SPRNG. This paper describes the reconfigurable logic of HASPRNG exploiting the mathematical properties and data parallelism residing in the SPRNG algorithms to produce high performance and also describes how to use the programming interface to minimize the communication overhead between FPGAs and microprocessors. The programming interface allows a user to be able to use HASPRNG the same way as SPRNG 2.0 on platforms such as the Cray XD1. We also describe how to install HASPRNG and use it. For HASPRNG usage we discuss a FPGA π-estimator for a High Performance Reconfigurable Computer (HPRC) sample application and compare to a software π-estimator. HASPRNG shows 1.7x speedup over SPRNG on the Cray XD1 and is able to obtain substantial speedup for a HPRC application. Program summary Program title: HASPRNG Catalogue identifier: AEER_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEER_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 594 928 No. of bytes in distributed program, including test data, etc.: 6 509 724 Distribution format: tar.gz Programming language: VHDL (XUP and Cray XD1), C++ (XUP), C (Cray XD1) Computer: PowerPC 405 (XUP) / AMD 2.2 GHz Opteron processor (Cray XD1) Operating system: Linux File size: 15 MB (XUP) / 22 MB (Cray XD1) Classification: 4.13 Nature of problem: Many computational science applications are able to consume large numbers of random numbers. For example, Monte Carlo simulations such as π-estimation are able to consume limitless random numbers forthe computation as long as hardware resources for the computing are supported. Moreover, parallel computational science applications require independent streams of random numbers to attain statistically significant results. The SPRNG library provides this capability, but at a significant computational cost. The library presented here accelerates the generators of independent streams of random numbers. Solution method: Multiple copies of random number generators in FPGAs allow a computational science application to consume large numbers of random numbers from independent, parallel streams. HASPRNG is a random number generators library to allow a computational science application to employ the multiple copies of random number generators to boost performance. Users can interface HASPRNG with software code executing on microprocessors and/or with hardware applications executing on FPGAs.

An Object-Oriented Random-Number Package with Many Long Streams and Substreams

Multiple independent streams of random numbers are often required in simulation studies, for instance, to facilitate synchronization for variance-reduction purposes, and for making independent replications. A portable set of software utilities is described for uniform random-number generation. It provides for multiple generators (streams) running simultaneously, and each generator (stream) has its sequence of numbers partitioned into many long disjoint contiguous substreams. The basic underlying generator for this implementation is a combined multiple-recursive generator with period length of approximately 2191, proposed by L'Ecuyer (1999a). A C++ interface is described here. Portable implementations are available in C, C++, and Java as supplemental material at http://dx.doi.org/10.1287/opre.50.6.1073.358

Simulation metamodel estimation using a combined correlation-based variance reduction technique for first and higher-order metamodels

This paper develops a procedure for jointly applying the correlation-based variance reduction techniques of common random numbers and control variates in a simulation experiment that is designed to estimate a linear metamodel (that is, linear in the regression model parameters) for a single response variable expressed in terms of an input vector of design variables for the target system. This procedure combines (a) the method of common random numbers for metamodel estimation and (b) a metamodel estimation scheme based on the method of control variates. Under specified conditions on the dependency structure of the simulation outputs and with respect to a variety of optimality criteria, the combined procedure is shown to be superior to each of the following conventional correlation-based variance reduction techniques: independent random number streams, common random number streams, and control variates.

Strategies for Combining Antithetic Variates and Control Variates in Designed Simulation Experiments

In this paper we examine three methods for combining the variance reduction techniques of antithetic variates and control variates to estimate the mean response in a designed simulation experiment. In Combined Method I, we perform h independent pairs of simulation runs as follows—on the second run of each such pair, we use random number streams that are antithetic (complementary) to the streams used on the first run of the pair to drive the non-control-variate components of the simulation model; and we use independent random number streams to drive the control-variate components of the simulation model. In Combined Method II, we also perform h independent pairs of runs; but on each pair of runs we use independent random number streams to drive the non-control-variate model components, and we use antithetic random number streams to drive the control-variate components. In Combined Method III, all of the random number streams driving the second run of each pair of runs are antithetic to the streams driving the first run of the pair. For each of these three methods we drive the variance of the resulting estimator of the mean response to make a theoretical comparison of the efficiency of each method. We implemented these three methods, along with the classical method of control variates, in a simulation model of a resource-constrained activity network to show how each combined method is implemented in practice and to evaluate the performance of each combined method experimentally. The results indicate that: (a) Combined Method III outperformed all other methods, and (b) the effectiveness of Combined Method III as well as the choice of whether to use Combined Method I or Combined Method II depends on the degree of correlation between the control variates and the response.

ESTIMATING SIMULATION METAMODELS USING COMBINED CORRELATION-BASED VARIANCE REDUCTION TECHNIQUES

This paper develops a procedure for jointly applying all of the correlation-based variance reduction techniques (namely, die methods of antithetic variates, common random numbers, and control variates) in a simulation experiment that is designed to estimate a linear metamodel (that is, a linear regression model) for a single response variable expressed in terms of an input vector of design variables for the target system. This procedure combines (a) the Schruben-Margolin strategy for metamodel estimation based on joint application of the methods of common random numbers and antithetic variates, and (b) a metamodel estimation scheme based on the method of control variates. Under specified conditions on the dependency structure of die simulation outputs and with respect to a variety of optimally criteria, the combined procedure is shown to be superior to each of the following conventional correlation-based variance reduction techniques: independent random number streams, common random number streams, contro...

An improved generation technique for random number sequences

This paper deals with the problem of producing several statistically independent streams of random numbers. Existing techniques are discussed and a new method is proposed. Statistical tests carried out on all the circuits described show that the proposed method yields random number sequences whose statistical independence is better than that of the sequences generated by existing techniques. The proposed designs can be easily implemented using MSI devices, and offers considerable speed with an economy of hardware.