The Effects of Unequal Covariances on the Tukey WSD Test

Abstract

The purpose of this study was to determine the effect of violating the assumption of homogeneity of covariance for the Tukey WSD test on the familywise rate of Type I error, when all other assumptions of the test were met. Determinations of the empirical significance level were made using computerized Monte Carlo simulations varying the number of treatment groups ( k = 3 and 5), sample size ( n = 5, 15, and 25), and degree of covariance heterogeneity (none, moderate, and high). For each of the resulting 18 combinations, 10,000 separate data sets were generated and analyzed at nominal α = .01 and .05 using the Tukey WSD procedure for all pairwise comparisons of means. For the conditions investigated in this study, degree of covariance heterogeneity showed a consistent effect on the empirical significance level. As the degree of covariance heterogeneity was increased, the empirical significance levels increased beyond the nominal levels. In general, different combinations of sample size and number of treatments produced no consistent effect on the empirical significance level.