A review of exact hypothesis testing procedures (and selection techniques) that control power regardless of the variances

Abstract

When testing hypotheses, two important problems that applied statisticians must consider are whether a large enough sample was used, and what to do when the frequently adopted homogeneity of variance assumption is violated. The first goal in this paper is to briefly review exact solutions to these problems. In the one‐way ANOVA, for example, these procedures tell an experimenter whether enough observations were sampled so that the power will be at least as large as some pre‐specified level. If too few observations were sampled, the procedure indicates how many more observations are required. The solution is exact, which is in contrast to another well‐known procedure described in the paper. Also, the variances are allowed to be unequal. The second goal is to review how the techniques used to test hypotheses have also been used to solve problems in selection.