Error Rates and Sample Sizes for Range Tests in Multiple Comparisons

Abstract

The problem of multiple comparisons has recently aroused a great deal of interest among statisticians. The basic F-test in an analysis of variance determines whether there is a significant difference among a group of means, but it cannot tell which means differ significantly from which others. The latter is often what the investigator really wants to know. Various multiple comparisons tests, including the range tests discussed in this paper, have been proposed. A study is made here of the error rates, a and 3, and their relation to sample size, N, for three fixed range tests and three multiple range tests. Let it be required to test the significance of the range of p out of m ordered means of samples of size N, where p = 2, 3, * *, m. For the fixed range tests, the critical range for a particular Type I error rate, a, depends only on m and N; for the multiple range tests, it depends also on p. In fact, for fixed m and N, it is a non-decreasing function of p. The range of p means is said to be significant or non-significant (at the a level) according as it does or does not exceed the critical range, except that the range of p means is automatically non-significant if these p means constitute a subgroup of a larger group whose range is non-significant. Because of this exception, a multiple range test starts with a test on all m means. If the range of all me means is found to be significant, then tests are performed on the ranges of (m -1) successive means, (m -2) successive means, and so on until significant differences are no longer found.