Maximum familywise Type I error rate: The least significant difference, Newman-Keuls, and other multiple comparison procedures.

Abstract

This article argues that the most reasonable and cautious definition of error rate in the multiple comparisons problem is the maximum familywise rate of Type I error (MFWER), that is, the maximum error rate attainable under all possible null hypotheses. This article shows how the original formulations of Fisher's least significant difference (LSD) and the Newman-Keuls procedures, which define the error rate with respect to only the complete null hypothesis, do not limit the MFWER to the level of significance. Modified LSD and Newman-Keuls procedures that do limit the MFWER are presented. Finally, additional multiple comparisons procedures that limits the MFWER and are more powerful than currently used tets are enumerated