Random Number Streams Research Articles

Random numbers for parallel computers: Requirements and methods, with emphasis on GPUs

We examine the requirements and the available methods and software to provide (or imitate) uniform random numbers in parallel computing environments. In this context, for the great majority of applications, independent streams of random numbers are required, each being computed on a single processing element at a time. Sometimes, thousands or even millions of such streams are needed. We explain how they can be produced and managed. We devote particular attention to multiple streams for GPU devices.

RNGAVXLIB: Program library for random number generation, AVX realization

Abstract The library RNGSSELIB for random number generators (RNGs) based upon the SSE2 command set is presented. The library contains realization of a number of modern and most reliable generators. Usage of SSE2 command set allows to substantially improve performance of the generators. Three new RNG realizations are also constructed. We present detailed analysis of the speed depending on compiler usage and associated optimization level, as well as results of extensive statistical testing for all generators using available test packages. Fast SSE implementations produce exactly the same output sequence as the original algorithms. Program summary Program title: RNGSSELIB Catalogue identifier: AEIT_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEIT_v1_0.html Program obtainable from: CPC Program Library, Queenʼs University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 4177 No. of bytes in distributed program, including test data, etc.: 21 228 Distribution format: tar.gz Programming language: C. Computer: PC. Operating system: UNIX, Windows. RAM: 1 Mbytes Classification: 4.13. Nature of problem: Any calculation requiring uniform pseudorandom number generator, in particular, Monte Carlo calculations. Solution method: The library contains realization of a number of modern and reliable generators: mt19937 , mrg32k3a and lfsr113 . Also new realizations for the method based on parallel evolution of an ensemble of dynamical systems are constructed: GM19 , GM31 and GM61 . The library contains both usual realizations and realizations based on SSE command set. Usage of SSE commands allows the performance of all generators to be substantially improved. Restrictions: For SSE realizations of the generators, Intel or AMD CPU supporting SSE2 command set is required. In order to use the realization lfsr113sse , CPU must support SSE4 command set. Running time: Running time is of the order of 20 sec for generating 109 pseudorandom numbers with a PC based on Intel Core i7-940 CPU. Running time is analysed in detail in Section 5 of the paper.

The Table-Hadamard GRNG

Gaussian random number generators (GRNGs) are an important component in parallel Monte Carlo simulations using FPGAs, where tens or hundreds of high-quality Gaussian samples must be generated per cycle using very few logic resources. This article describes the Table-Hadamard generator, which is a GRNG designed to generate multiple streams of random numbers in parallel. It uses discrete table distributions to generate pseudo-Gaussian base samples, then a parallel Hadamard transform to efficiently apply the central limit theorem. When generating 64 output samples, the Table-Hadamard requires just 130 slices per generated sample, which is a third of the resources needed by the next best technique, while still providing higher statistical quality.

Hybrid pseudo-random number generator for cryptographic systems

For a powerful cryptographic system, high-quality random number streams are essential. Those raw pseudo-random number generators (PRNG) that are used to generate high-quality random numbers have some disadvantages, such as failure to meet the R4 security requirement. Therefore, use of random number sequences generated by these generators in a cryptographic system puts the entire system at risk. This study proposes a new hybrid PRNG by means of an additional input introduced to transition and output functions used in a raw PRNG system in order to eliminate this risk. The additional inputs to the designed system have been implemented via the true random number generator developed by using the Sprott 94 G chaotic system on FPGA. The random number streams obtained from the recommended hybrid structure have been subjected to the NIST 800.22 and FIPS statistical test, which have given good results. According to these results, it has been proved that the recommended hybrid PRNG system meets the R4 security requirement and can be used in cryptographic applications.

Implementation and validation of a Wilks-type multi-site daily precipitation generator over a typical Alpine river catchment

Abstract. Many climate impact assessments require high-resolution precipitation time series that have a spatio-temporal correlation structure consistent with observations, for simulating either current or future climate conditions. In this respect, weather generators (WGs) designed and calibrated for multiple sites are an appealing statistical downscaling technique to stochastically simulate multiple realisations of possible future time series consistent with the local precipitation characteristics and their expected future changes. In this study, we present the implementation and validation of a multi-site daily precipitation generator re-built after the methodology described in Wilks (1998). The generator consists of several Richardson-type WGs run with spatially correlated random number streams. This study aims at investigating the capabilities, the added value and the limitations of the precipitation generator for a typical Alpine river catchment in the Swiss Alpine region under current climate. The calibrated multi-site WG is skilful at individual sites in representing the annual cycle of the precipitation statistics, such as mean wet day frequency and intensity as well as monthly precipitation sums. It reproduces realistically the multi-day statistics such as the frequencies of dry and wet spell lengths and precipitation sums over consecutive wet days. Substantial added value is demonstrated in simulating daily areal precipitation sums in comparison to multiple WGs that lack the spatial dependency in the stochastic process. Limitations are seen in reproducing daily and multi-day extreme precipitation sums, observed variability from year to year and in reproducing long dry spell lengths. Given the performance of the presented generator, we conclude that it is a useful tool to generate precipitation series consistent with the mean climatic aspects and likely helpful to be used as a downscaling technique for climate change scenarios.

Improving the Efficiency and Efficacy of Stochastic Trust-Region Response-Surface Method for Simulation Optimization

Stochastic Trust-Region Response-Surface method (STRONG) is a new response-surface-based framework for simulation optimization. The appeal of STRONG lies in that it preserves the advantages, yet eliminates the disadvantages, of traditional response surface methodology (RSM) that has been used for more than 50 years. Specifically, STRONG does not require human involvement in the search process and can guarantee to converge to the true optimum with probability one (w.p.1). In this paper, we propose an improved framework, called STRONG-X, that enhances the efficiency and efficacy of STRONG to widen its applicability to more practical problems. For efficiency improvement, STRONG-X includes a newly-developed experimental scheme that consists of construction of optimal simulation designs and an assignment strategy for random number streams to obtain computational gains. For efficacy improvement, a new variant, called STRONG-XG, is developed to achieve convergence under generally-distributed responses, as opposed to STRONG and STRONG-X where convergence is guaranteed only when the response is normal. An extensive numerical study is conducted to evaluate the efficiency and efficacy of STRONG-X and STRONG-XG. Moreover, two illustrative examples are provided to show the viability of STRONG-X and STRONG-XG in practical settings.

Generation of Random Numbers and Parallel Random Number Streams for Monte Carlo Simulations

Modern methods and libraries for high quality pseudorandom number generation and for generation of parallel random number streams for Monte Carlo simulations are considered. The probability equidistribution property and the parameters when the property holds at dimensions up to logarithm of mesh size are considered for Multiple Recursive Generators.

On parallel random number generation for accelerating simulations of communication systems

Abstract. Powerful compute clusters and multi-core systems have become widely available in research and industry nowadays. This boost in utilizable computational power tempts people to run compute-intensive tasks on those clusters, either for speed or accuracy reasons. Especially Monte Carlo simulations with their inherent parallelism promise very high speedups. Nevertheless, the quality of Monte Carlo simulations strongly depends on the quality of the employed random numbers. In this work we present a comprehensive analysis of state-of-the-art pseudo random number generators like the MT19937 or the WELL generator used for parallel stream generation in different settings. These random number generators can be realized in hardware as well as in software and help to accelerate the analysis (or simulation) of communications systems. We show that it is possible to generate high-quality parallel random number streams with both generators, as long as some configuration constraints are met. We furthermore depict that distributed simulations with those generator types are viable even to very high degrees of parallelism.

Finite Element Analysis of the Microstructure–Strength Relationships of Metal Matrix Composites

The influence of the shape and spatial distribution of reinforced particles on strength and damage of metal matrix composite (MMC) is investigated through finite element method under uniaxial tensile, simple shear, biaxial tensile, as well as combined tensile/shear loadings. The particle shapes change randomly from circular to regular n-sided polygon (3 ≤ n ≤ 10); the particle alignments are determined through a sequentially random number stream and the particle locations are defined through the random sequential adsorption algorithm. The ductile failure in metal matrix and brittle failure in particles are described through damage models based on the stress triaxial indicator and maximum principal stress criterion, respectively, while the debonding behavior of interface between particles and matrix is simulated through cohesive elements. The simulation results show that, under different loadings, interface debonding is the dominated failure mechanism in MMCs and plastic deformation and ductile failure of matrix also play very important roles on the failure of MMCs.

Using RngStreams for parallel random number generation in C++ and R

The RngStreams software package provides one viable solution to the problem of creating independent random number streams for simulations in parallel processing environments. Techniques are presented for effectively using RngStreams with C++ programs that are parallelized via OpenMP or MPI. Ways to access the backbone generator from RngStreams in R through the parallel and rstream packages are also described. The ideas in the paper are illustrated with both a simple running example and a Monte Carlo integration application.

PRAND: GPU accelerated parallel random number generation library: Using most reliable algorithms and applying parallelism of modern GPUs and CPUs

The library PRAND for pseudorandom number generation for modern CPUs and GPUs is presented. It contains both single-threaded and multi-threaded realizations of a number of modern and most reliable generators recently proposed and studied in Barash (2011), Matsumoto and Tishimura (1998), L’Ecuyer (1999,1999), Barash and Shchur (2006) and the efficient SIMD realizations proposed in Barash and Shchur (2011). One of the useful features for using PRAND in parallel simulations is the ability to initialize up to 1019 independent streams. Using massive parallelism of modern GPUs and SIMD parallelism of modern CPUs substantially improves performance of the generators. Program summaryProgram title: PRANDCatalogue identifier: AESB_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AESB_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 45979No. of bytes in distributed program, including test data, etc.: 23953564Distribution format: tar.gzProgramming language: Cuda C, Fortran.Computer: PC, workstation, laptop, or server with NVIDIA GPU (tested on Tesla X2070, Fermi C2050, GeForce GT540M) and with Intel or AMD processor.Operating system: Linux with CUDA version 5.0 or later. Should also run on MacOs, Windows, or UNIX.RAM: 4 MbytesClassification: 4.13.Nature of problem:Any calculation requiring uniform pseudorandom number generator, in particular, Monte Carlo calculations. Any calculation or simulation requiring uncorrelated parallel streams of uniform pseudorandom numbers.Solution method:The library contains realization of a number of modern and reliable generators: MT19937, MRG32K3A and LFSR113. Also new realizations of the method based on parallel evolution of an ensemble of transformations of two-dimensional torus are included in the library: GM19, GM29, GM31, GM61, GM55, GQ58.1, GQ58.3 and GQ58.4. The library contains: single-threaded and multi-threaded realizations for GPU, single-threaded realizations for CPU, realizations for CPU based on SSE command set. Also, the library contains the abilities to jump ahead inside RNG sequence and to initialize independent random number streams with block splitting method for each of the RNGs.Restrictions:Nvidia Cuda Toolkit version 5.0 or later should be installed. To use GPU realizations, Nvidia GPU supporting CUDA and the corresponding Nvidia driver should be installed. For SSE realizations of the generators, Intel or AMD CPU supporting SSE2 command set is required. In order to use the SSE realization of LFSR113, CPU must support SSE4 command set.Additional comments:A version of this program, which only contains the realizations for CPUs, is held in the Library as Catalog Id., AEIT_v2_0 (RNGSSELIB). It does not require a GPU device or CUDA compiler.Running time:The tests and the examples included in the package take less or about one minute to run. Running time is analyzed in detail in Section 8 of the paper.

RNGSSELIB: Program library for random number generation. More generators, parallel streams of random numbers and Fortran compatibility

In this update, we present the new version of the random number generator (RNG) library RNGSSELIB, which, in particular, contains fast SSE realizations of a number of modern and most reliable generators \cite{RNGSSELIB1}. The new features are: i) Fortran compatibility and examples of using the library in Fortran; ii) new modern and reliable generators; iii) the abilities to jump ahead inside RNG sequence and to initialize up to $10^{19}$ independent random number streams with block splitting method.

Variance Reduction Techniques on Generating M/M/1 Processes in Simulation Output Analysis

Observations from steady-state M/M/1 processes are often used by simulation-methodology researchers conducting Monte Carlo performance evaluations of output analysis, variance reduction, and optimization. The classical method for generating data from M/M/1 processes requires two random number streams. Motivated by the need for variance reduction in simulation experiments, we propose inverse-transformation algorithms for generating M/M/1 processes, including waiting times in queue (M/M/1-QT) and in system (M/M/1-ST), using a single random number stream. Moreover, we propose analytical results for computing the variance of the sample mean from M/M/1 processes. The proposed method for generating processes for the M/M/1-QT and the M/M/1-ST is superior (in terms of statistical effectiveness; i.e., variance reduction) to the corresponding classical methods using two random number streams. Moreover, the proposed generation method for M/M/1-QT is also superior to the classical method in terms of computational efficiency. Finally, although the proposed M/M/1-ST generation method is not as computationally efficient as the classical technique, it does have the same computational efficiency as the classical method if a parallel computing scheme is adopted with the Newton's search method.

Interval-Based Simulation to Model Input Uncertainty in Stochastic Lanchester Models

Interval-based simulation (IBS) has been proposed to model input uncertainty in discrete-event simulation. The foundation of this new simulation paradigm is imprecise probability, which models systems under both aleatory and epistemic uncertainties. The statistical distribution parameters in IBS are represented by intervals instead of precise real numbers. This paper discusses how the IBS approach can be applied to stochastic Lanchester models that are used in combat simulation to better account for input parameter uncertainty. The advantages of this approach are explored in comparison with second-order Monte Carlo simulation. Using IBS, an improved estimate of the probability of a team winning a battle is calculated by taking advantage of the interval structure. By resampling from intervals to determine event times, we can separate the effect of parameter uncertainty from random number generator uncertainty to estimate the probability that one team will win for given stream of random numbers used in a single replication. Additionally, we show how our method can be used to improve the reliability of stochastic Lanchester results by accounting for different skill levels within each team, and show how the interval structure can be used to highlight the disproportionate effect of the first few encounters in the battle.

Variants of Mersenne Twister Suitable for Graphic Processors

This article proposes a type of pseudorandom number generator, Mersenne Twister for Graphic Processor (MTGP), for efficient generation on graphic processessing units (GPUs). MTGP supports large state sizes such as 11213 bits, and uses the high parallelism of GPUs in computing many steps of the recursion in parallel. The second proposal is a parameter-set generator for MTGP, named MTGP Dynamic Creator (MTGPDC). MTGPDC creates up to 2 32 distinct parameter sets which generate sequences with high-dimensional uniformity. This facility is suitable for a large grid of GPUs where each GPU requires separate random number streams. MTGP is based on linear recursion over the two-element field, and has better high-dimensional equidistribution than the Mersenne Twister pseudorandom number generator.

A stochastic programming approach for sawmill production planning

This paper investigates a sawmill production planning problem where the non-homogeneous characteristics of logs result in random process yields. A two-stage stochastic Linear Programming (LP) approach is proposed to address this problem. The random yields are modelled as scenarios with discrete probability distributions. The solution methodology is based on the sample average approximation method. Confidence intervals are constructed for the optimality gap of several candidate solutions, based on Common Random Number (CRN) streams. A computational study including a prototype sawmill is presented to highlight the significance of using the stochastic model instead of the mean-value deterministic model, which is the traditional production planning tool in sawmills.

GASPRNG: GPU accelerated scalable parallel random number generator library

Graphics processors represent a promising technology for accelerating computational science applications. Many computational science applications require fast and scalable random number generation with good statistical properties, so they use the Scalable Parallel Random Number Generators library (SPRNG). We present the GPU Accelerated SPRNG library (GASPRNG) to accelerate SPRNG in GPU-based high performance computing systems. GASPRNG includes code for a host CPU and CUDA code for execution on NVIDIA graphics processing units (GPUs) along with a programming interface to support various usage models for pseudorandom numbers and computational science applications executing on the CPU, GPU, or both. This paper describes the implementation approach used to produce high performance and also describes how to use the programming interface. The programming interface allows a user to be able to use GASPRNG the same way as SPRNG on traditional serial or parallel computers as well as to develop tightly coupled programs executing primarily on the GPU. We also describe how to install GASPRNG and use it. To help illustrate linking with GASPRNG, various demonstration codes are included for the different usage models. GASPRNG on a single GPU shows up to 280x speedup over SPRNG on a single CPU core and is able to scale for larger systems in the same manner as SPRNG. Because GASPRNG generates identical streams of pseudorandom numbers as SPRNG, users can be confident about the quality of GASPRNG for scalable computational science applications. Program summaryProgram title: GASPRNGCatalogue identifier: AEOI_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEOI_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: UTK license.No. of lines in distributed program, including test data, etc.: 167900No. of bytes in distributed program, including test data, etc.: 1422058Distribution format: tar.gzProgramming language: C and CUDA.Computer: Any PC or workstation with NVIDIA GPU (Tested on Fermi GTX480, Tesla C1060, Tesla M2070).Operating system: Linux with CUDA version 4.0 or later. Should also run on MacOS, Windows, or UNIX.Has the code been vectorized or parallelized?: Yes. Parallelized using MPI directives.RAM: 512 MB∼ 732 MB (main memory on host CPU, depending on the data type of random numbers.) / 512 MB (GPU global memory)Classification: 4.13, 6.5.Nature of problem:Many computational science applications are able to consume large numbers of random numbers. For example, Monte Carlo simulations are able to consume limitless random numbers for the computation as long as resources for the computing are supported. Moreover, parallel computational science applications require independent streams of random numbers to attain statistically significant results. The SPRNG library provides this capability, but at a significant computational cost. The GASPRNG library presented here accelerates the generators of independent streams of random numbers using graphical processing units (GPUs).Solution method:Multiple copies of random number generators in GPUs allow a computational science application to consume large numbers of random numbers from independent, parallel streams. GASPRNG is a random number generators library to allow a computational science application to employ multiple copies of random number generators to boost performance. Users can interface GASPRNG with software code executing on microprocessors and/or GPUs.Running time:The tests provided take a few minutes to run.

Distribution of random streams for simulation practitioners

SUMMARYThere is an increasing interest in the distribution of parallel random number streams in the high‐performance computing community particularly, with the manycore shift. Even if we have at our disposal statistically sound random number generators according to the latest and thorough testing libraries, their parallelization can still be a delicate problem. Indeed, a set of recent publications shows it still has to be mastered by the scientific community. With the arrival of multi‐core and manycore processor architectures on the scientist desktop, modelers who are non‐specialists in parallelizing stochastic simulations need help and advice in distributing rigorously their experimental plans and replications according to the state of the art in pseudo‐random numbers parallelization techniques. In this paper, we discuss the different partitioning techniques currently in use to provide independent streams with their corresponding software. In addition to the classical approaches in use to parallelize stochastic simulations on regular processors, this paper also presents recent advances in pseudo‐random number generation for general‐purpose graphical processing units. The state of the art given in this paper is written for simulation practitioners. Copyright © 2012 John Wiley & Sons, Ltd.

Parallel generation of 10 Gbits/s physical random number streams using chaotic semiconductor lasers

In this paper, we have realized experimentally, for the first time to our knowledge, a parallel gen� eration scheme of high speed random numbers by using chaotic laser. In our setup, two broadband chaos sig� nals are obtained experimentally from a mutually coupled semiconductor lasers system. The bandwidths of these signals are over 10 GHz. Next, by using such chaos signals as entropy sources, two 10 Gbits/s physical random number streams are generated concurrently and successfully pass the NIST statistical tests. Our sys� tem is scalable and the rate of random numbers could be further enhanced by simply adding more channels.

Random number generators for massively parallel simulations on GPU

High-performance streams of (pseudo) random numbers are crucial for the efficient implementation for countless stochastic algorithms, most importantly, Monte Carlo simulations and molecular dynamics simulations with stochastic thermostats. A number of implementations of random number generators has been discussed for GPU platforms before and some generators are even included in the CUDA supporting libraries. Nevertheless, not all of these generators are well suited for highly parallel applications where each thread requires its own generator instance. For this specific situation encountered, for instance, in simulations of lattice models, most of the high-quality generators with large states such as Mersenne twister cannot be used efficiently without substantial changes. We provide a broad review of existing CUDA variants of random-number generators and present the CUDA implementation of a new massively parallel high-quality, high-performance generator with a small memory load overhead.