Statistical inference, Type II error, and decision making under the US Endangered Species Act

Abstract

Critical conservation decisions have been made based on the spurious belief that “no statistically significant difference between two groups means the groups are the same”. We demonstrate this using the case of the Preble's meadow jumping mouse (Zapus hudsonius preblei), an endangered species in the US. Such faulty statistical logic has been recognized before, but ecologists have typically recommended assessing post hoc statistical power as a remedy. Statisticians, however, have shown that observed power will necessarily be low when no differences are found between two populations. Alternatives to assessments of statistical power include equivalence testing (a method rarely used by ecologists) and Bayesian or likelihood methods. Although scientists play a central role in ameliorating this problem, the courts could also assist by requiring litigated federal agency decisions to consider the risks of both Type I and Type II errors.