Selecting Significant Effects in Factorial Designs Taking Type II Errors into Account

Abstract

When analysing the effects of a factorial design, it is customary to take into account the probability of making a Type I error (the probability of considering an effect significant when it is non-significant), but not to consider the probability of making a Type II error (the probability of considering an effect as non-significant when it is significant). Making a Type II error, however, may lead to incorrect decisions regarding the values that the factors should take or how subsequent experiments should be conducted. In this paper, we introduce the concept of minimum effect size of interest and present a visualization method for selecting the critical value of the effects, the threshold value above which an effect should be considered significant, which takes into account the probability of Type I and Type II errors. Copyright © 2006 John Wiley & Sons, Ltd.