Power of experimental designs, estimated by Monte Carlo simulation

Abstract

What is the optimal size of an experiment? How should the practical experimenter determine this optimal experimental size? The paper presents a conceptually simple method for estimating the statistical power of experimental designs, based on Monte Carlo simulation. In the planning stage of a project, several alternative experimental designs may thereby be compared with respect to their ability to balance the risk of committing Type I and Type II errors against the cost. The Monte Carlo power estimation of a design is based on the following main steps: generate artificial data in a number of (say 5000) hypothetical experiments, based on the design and on certain assumptions; analyse each artificial data set in the same way that the future, real data set is intended to be analysed; from the distributions of the obtained parameter estimates, study the risks associated with the given experimental design. The method is illustrated for a factorial design in sensory analysis concerning warmed-over flavour development in meat. Here the Monte Carlo simulations indicated that four replicates were needed, given certain assumptions. The real experiment was performed independently twice, each time in four replicates. The resulting analysis of effects gave satisfactory results, indicating that four replicates had given the necessary and sufficient power in each of the two experiments. Copyright © 2000 John Wiley & Sons, Ltd.