Power of pairwise comparisons in the equal variance and unequal sample size case

Abstract

A Monte Carlo simulation was conducted to compare five, pairwise multiple comparison procedures. The number of means varied from 4 to 6 and the sample size ratio varied from 1 to 60. Procedures were evaluated on the basis of Type I errors, any-pair power and all-pairs power. Four procedures were shown to be conservative, while the fifth provided adequate control of Type I errors only for restricted values of sample size ratios. No procedure was found to be uniformly most powerful. The Tukey-Kramer procedure was found to provide the best any-pair power provided it is applied without requiring a significant overall F test. In most cases, the Hayter-Fisher modification of the Tukey-Kramer was found to provide very good any-pair power and to be uniformly more powerful than the Tukey-Kramer when a significant overall F test is required. A partition-based version of Peritz's method usually provided the greatest all-pairs power. A modification of the Shaffer-Welsch was found to be useful in certain conditions.