Evaluating Stream Habitat Survey Data and Statistical Power Using an Example from Southeast Alaska

Abstract

Stream habitat surveys and watershed assessments have been developed and used as monitoring tools for decades. Most rely on type I error as the primary criterion, with minor consideration of statistical power and effect size. We test for statistical differences in fish habitat condition between harvested and nonharvested watersheds from habitat survey data collected in southeast Alaska. We apply statistical power analysis to judge whether nonsignificant results can be interpreted with confidence. None of the fish habitat variables we examined were significant at α = 0.05; however, several P-values were less than 0.10 and consistent differences between harvested and nonharvested reaches were observed among channel types. Statistical power is low and the probability of not detecting differences is high when the effect size, scaled to the standard deviation of the measurement, is small to medium. For large effect sizes, the ability to detect differences was greater but did not exceed 85% for any measurement. Statistical power, effect size, and biological significance of the outcome are important considerations when the results are interpreted and can lend additional information to managers making decisions with data that are less than perfect.