Type I error rates, power, and sample sizes for two-stage solutions to the Behrens–Fisher problem when population distributions are non-normal

Abstract

Two-stage sampling procedures for comparing two population means developed by Chapman (1950) and Ghosh (1975) are compared in terms of Type I errors, power, and sample size requirements when populations are non-normal. The Ghosh procedure is shown to be less sensitive to non-normal distributions but can have actual Type I error rates greater than the nominal when sampling from distributions that are skewed and the initial sample size is small. Moderate to large sample sizes at the first sampling stage can reduce the overall total sample size needed and can minimize the inflated Type I error rate. Average sample sizes needed remain constant across distribution shapes but greater variability is found with heavy-tailed distributions. Total sample sizes needed for the Ghosh procedure were estimated for a variety of effect sizes and are compared with the Student t-test when all parametric assumptions, including equal population variances, are met. Only small differences in the sample sizes are found when the initial sample size is at least moderate.