Replication, prep, and confidence intervals: Comment prompted by Iverson, Wagenmakers, and Lee (2010); Lecoutre, Lecoutre, and Poitevineau (2010); and Maraun and Gabriel (2010).

Abstract

This comment offers three descriptions of prep that start with a frequentist account of confidence intervals, draw on R. A. Fisher's fiducial argument, and do not make Bayesian assumptions. Links are described among prep, p values, and the probability a confidence interval will capture the mean of a replication experiment. The descriptions suggest the criticism of Maraun and Gabriel (2010) is unjustified. Iverson, Wagenmakers, and Lee (2010) discussed prep in terms of Bayesian model averaging. This went usefully beyond the dichotomous decision making of significance testing, but an extension to Bayesian estimation would be welcome. Lecoutre, Lecoutre, and Poitevineau (2010) referred to and extended their substantial research based on predictive approaches. Some of the links they make among p values, confidence intervals, and prep parallel links described earlier, although their conceptual framework is different. The interesting prep experiment in Psychological Science may be coming to a close; it suggests that statistical innovation, including that proposed by Iverson et al. (2010) and Lecoutre et al. (2010), is likely to be most successful if guided by cognitive evidence and supported by resources tailored for researchers generally.