Use and Misuse of Multiple Comparison Procedures1

Abstract

AbstractA number of statistical tests can assist in the interpretation of the results of agronomic experiments. Examination of recent issues of the Agronomy Journal indicates that one of the more popular procedures is to use some form of a multiple comparison test to look at pairwise comparisons among the treatment means. In this paper use of these tests is examined, and alternative procedures are suggested for situations in which multiple comparison tests are inappropriate. These are illustrated with examples extracted from the Agronomy Journal. In general, multiple comparison procedures are appropriate for “data snooping” following experiments with unstructured qualitative treatments. Multiple comparison tests are almost never appropriate for experiments in which 1) the treatments are graded levels of a quantitative variable, 2) for factorial combinations of two or more factors at two or more levels, and 3) for quantitative treatments where previously formulated linear combinations of treatment means are of particular interest.