Abstract
According to a recent meta-analysis, religious priming has a positive effect on prosocial behavior (Shariff et al., 2015). We first argue that this meta-analysis suffers from a number of methodological shortcomings that limit the conclusions that can be drawn about the potential benefits of religious priming. Next we present a re-analysis of the religious priming data using two different meta-analytic techniques. A Precision-Effect Testing–Precision-Effect-Estimate with Standard Error (PET-PEESE) meta-analysis suggests that the effect of religious priming is driven solely by publication bias. In contrast, an analysis using Bayesian bias correction suggests the presence of a religious priming effect, even after controlling for publication bias. These contradictory statistical results demonstrate that meta-analytic techniques alone may not be sufficiently robust to firmly establish the presence or absence of an effect. We argue that a conclusive resolution of the debate about the effect of religious priming on prosocial behavior – and about theoretically disputed effects more generally – requires a large-scale, preregistered replication project, which we consider to be the sole remedy for the adverse effects of experimenter bias and publication bias.
Highlights
We argue that a conclusive resolution of the debate about the effect of religious priming on prosocial behavior – and about theoretically disputed effects more generally – requires a large-scale, preregistered replication project, which we consider to be the sole remedy for the adverse effects of experimenter bias and publication bias
To establish the robustness of psychological effects without the confounding influence of such biases, we argue for the use of large-scale and preregistered replication attempts, as facilitated for instance by the Open Science Framework (OSF)
The authors note that almost all meta-analyses show a similar negative relation between sample size and effect size; to us, this suggests that the results are contaminated by publication bias and/or questionable research practices (QRPs; e.g., optional stopping when the p-value approaches 0.05)
Summary
According to a recent meta-analysis, religious priming has a positive effect on prosocial behavior (Shariff et al, 2015). A PrecisionEffect Testing–Precision-Effect-Estimate with Standard Error (PET-PEESE) meta-analysis suggests that the effect of religious priming is driven solely by publication bias.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have