Absolute precision confidence intervals for unstandardized mean differences using sequential stopping rules.

Abstract

Societies and journals in psychology encourage use of confidence intervals (CIs) on effect sizes. Gaining a maximum of precision of the CI at a minimum cost is desirable. Methods are available to calculate a sample size to provide some percent "assurance" that the final CI will be no wider than a desired value under the fixed-sample rule (FSR), in which a sample size must be decided a priori. Such assurance is expensive, and still subject to failure. The desired width can be specified either in standardized or unstandardized units, and this article focuses on unstandardized widths. A sequential stopping rule (SSR) can generate a CI for a mean difference that is always the desired width, and the average use of subjects is about the same as the FSR sample size without assurance. Improper use of sequential sampling can lead to a CI that has degraded coverage - the interval may contain the population value only 90% of the time for a nominal 95% CI. SSR methods are available to deliver CIs that are within a certain tolerance of the nominal coverage or that are at least the nominal coverage on average. These methods can be assessed by simulations, and they involve a manipulation of the minimum sample size, the desired width, or the practical confidence coefficient used to calculate CIs during the SSR experiment. As with FSR, more exact CIs are generated when the population standard deviation can be estimated accurately.