Arguments for rejecting the sequential Bonferroni in ecological studies

Abstract

Interpretation of results that include multiple statistical tests has been an issue of great concern for some time in the ecological literature. The basic problem is that when multiple tests are undertaken, each at the same significance level ( ), the probability of achieving at least one significant result is greater than that significance level (Zaykin et al. 2002). Therefore, there is an increased probability of rejecting a null hypothesis when it would be inappropriate to do so. The typical solution to this problem has been lowering the values for the table (i.e. establishing a table-wide significance level) and therefore reducing the probability of a spurious result. Specifically, the most common procedure has been the application of the sequential Bonferroni adjustment (Holm 1979, Miller 1981, Rice 1989). Arguments in this essay address the problems of adjusting probability values for tables of multiple statistical tests, and more specifically argue for rejection of the sequential Bonferroni as a solution to this problem. Since the influential publication of Rice (1989), the sequential Bonferroni correction has become the primary method of addressing the problem of multiple statistical tests in ecological research. The sequential Bonferroni adjusts the table-wide p-value to keep it constant at 0.05, and subsequently reduces the probability of a spurious result. Although other methods exist for addressing tables of multiple statistical tests, the sequential Bonferroni has become the most commonly utilized process. However, this method has several flaws ranging from mathematical to logical to practical that argue for rejecting this method in ecological studies.