Abstract
The conventional form of statistical simulation proceeds by selecting a few models and generating hundreds or thousands of data sets from each model. This article investigates a different approach, called BayesSim, that generates hundreds or thousands of models from a prior distribution, but only one (or a few) data sets from each model. Suppose that the performance of estimators in a parametric model is of interest. Smoothing methods can be applied to BayesSim output to investigate how estimation error varies as a function of the parameters. In this way inferences about the relative merits of the estimators can be made over essentially the entire parameter space, as opposed to a few parameter configurations as in the conventional approach. Two examples illustrate the methodology: One involving the skew-normal distribution and the other nonparametric goodness-of-fit tests.
Highlights
When investigating statistical methodology using simulation, the following strategy is almost always used
Given BayesSim output, data could be categorized according to values of the loss function, and a support vector machine used to identify subsets of the regressor space that provide the best discrimination among these categories
Since we are interested in power, a 0 - 1 loss function is used, in which case the Bayes risk of each test is estimated by the proportion of rejections among all ten thousand replications
Summary
When investigating statistical methodology using simulation, the following strategy is almost always used. The idea is to generate one data set (or at most a few data sets) from each of hundreds or thousands of models that are randomly selected from a prior We call such an approach BayesSim. Given an appropriate loss function, one may generate BayesSim output to estimate the Bayes risk of each method of interest and use these estimates as at least part of a basis for choosing amongst methods. One may compare the conventional and BayesSim approaches by asking how efficiently each one can estimate R ( ⋅ ) using its information Another reason that BayesSim seems like a good idea is that it has the potential of simulating the average performance of a method that is used at a variety of times and places throughout the world. In comparing estimation methods for multiparameter models, one could estimate performance measures as a function of the parameter vector by using additive nonparametric regression methods
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
