Year-end Sale % OFF 
Abstract
In the current study, we introduce the prior predictive p-value as a method to test replication of an analysis of variance (ANOVA). The prior predictive p-value is based on the prior predictive distribution. If we use the original study to compose the prior distribution, then the prior predictive distribution contains datasets that are expected given the original results. To determine whether the new data resulting from a replication study deviate from the data in the prior predictive distribution, we need to calculate a test statistic for each dataset. We propose to use ̄F, which measures to what degree the results of a dataset deviate from an inequality constrained hypothesis capturing the relevant features of the original study: HRF. The inequality constraints in HRF are based on the findings of the original study and can concern, for example, the ordering of means and interaction effects. The prior predictive p-value consequently tests to what degree the new data deviates from predicted data given the original results, considering the findings of the original study. We explain the calculation of the prior predictive p-value step by step, elaborate on the topic of power, and illustrate the method with examples. The replication test and its integrated power and sample size calculator are made available in an R-package and an online interactive application. As such, the current study supports researchers that want to adhere to the call for replication studies in the field of psychology.
Highlights
New studies conducted to replicate earlier original studies are often referred to as replication studies
In the current study, we introduce the prior predictive p-value as a method to test replication of an analysis of variance (ANOVA)
Our method addresses a question similar to that in Anderson and Maxwell (2016), Harms (2018), Ly et al (2018), Verhagen and Wagenmakers (2014) and Patil et al (2016), but enables researchers to evaluate the replication of relevant features of an original ANOVA study
Summary
New studies conducted to replicate earlier original studies are often referred to as replication studies. Methodology on conducting replication studies has received increasing attention (see for example Anderson and Maxwell, 2016; Asendorpf et al, 2013; Brandt et al, 2014; Schmidt, 2009). Vote-counting is assessing whether the new effect is statistically significant and in the same direction as the significant effect in the original study (Anderson & Maxwell, 2016; Simonsohn, 2015). It is a dichotomous evaluation that does not take into account the magnitude of differences between effect-sizes of the original and new study (Asendorpf et al, 2013; Simonsohn, 2015). Underpowered replication studies are less likely to replicate significance, which can lead to misleading conclusions (Asendorpf et al, 2013; Cumming, 2008; Hedges & Olkin, 1980; Simonsohn, 2015)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
