Controlling the Type I error rate by using the nonparametric bootstrap when comparing means

Abstract

Of the several tests for comparing population means, the best known are the ANOVA, Welch, Brown-Forsythe, and James tests. Each performs appropriately only in certain conditions, and none performs well in every setting. Researchers, therefore, have to select the appropriate procedure and run the risk of making a bad selection and, consequently, of erroneous conclusions. It would be desirable to have a test that performs well in any situation and so obviate preliminary analysis of data. We assess and compare several tests for equality of means in a simulation study, including non-parametric bootstrap techniques, finding that the bootstrap ANOVA and bootstrap Brown-Forsythe tests exhibit a similar and exceptionally good behaviour.