A review of TESTU01

Abstract

The scientific literature is filled with examples of bad random number generators (RNGs), and a bad RNG can completely ruin a researcher’s analysis. See Coddington (1994, 1996) for examples and discussion from the field of physics. Examples from economics are difficult to find, in part, because many economists do not bother to test the RNGs that they use in their applied work. This may be due to the fact that they are unaware of any need to test the RNG. Econometrics texts simply assume (incorrectly) that RNGs work perfectly as advertised, despite the fact that many RNGs found in commercial software routinely fail randomness tests (L’Ecuyer, 1997, 2001). This is all the more shocking, since simulation results depend on the quality of the random numbers used as input—though one would hardly know this from reading books on simulation (e.g., Gourieroux and Montfort, 1996). If experts and textbooks fail to address the issue, the practicing economist can hardly be faulted. Even among economists who are aware of the issue, there is a hesitancy to test RNGs due a common sentiment: ‘I would probably just find that my RNGs fail one or more obscure tests, and I would have no idea of how to interpret such failures.’ Such a sentiment is understandable, but it misconstrues the purpose of randomness testing. As a practical matter, it is the province of the RNG expert, and not the economist, to interpret these failures. The economist’s job is simply to avoid RNGs that fail tests—not all tests, for there is no RNG that will pass all tests. The economist should avoid RNGs that fail ‘reasonable’ tests. What constitutes a reasonable test has been succinctly stated by L’Ecuyer (2004):