Statistical testing when the populations from which samples are drawn are uncertain

Abstract

The topic of this article is hypothesis testing when the populations from which the data are drawn are known only with a given probability distribution. Some important areas of application for which such a situation arises is reviewed briefly. The specific cases herein considered are testing a one-sided hypothesis involving two populations. An illustrative small data set, involving six observations, is used to demonstrate relevant approaches and calculations for such testing. Both a frequentist approach and a Bayesian approach are developed. In both of these approaches, use is made of all possible data configurations along with their corresponding probabilities. Various measures of goodness are developed for each of the two approaches. A simulation approach is developed for larger data sets.