Some cautions regarding statistical power in split-plot designs

Abstract

We show that if overall sample size and effect size are held constant, the power of theF test for a one-way analysis of variance decreases dramatically as the number of groups increases. This reduction in power is even greater when the groups added to the design do not produce treatment effects. If a second independent variable is added to the design, either a split-plot or a completely randomized design may be employed. For the split-plot design, we show that the power of theF test on the betweengroups factor decreases as the correlation across the levels of the within-groups factor increases. The attenuation in between-groups power becomes more pronounced as the number of levels of the withingroups factor increases. Sample size and total cost calculations are required to determine whether the split-plot or completely randomized design is more efficient in a particular application. The outcome hinges on the cost of obtaining (or recruiting) a single subject relative to the cost of obtaining a single observation: We call this thesubject-to-observation cost (SOC) ratio. Split-plot designs are less costly than completely randomized designs only when the SOC ratio is high, the correlation across the levels of the within-groups factor is low, and the number of such levels is small.