Professors are Not Elderly: Evaluating the Evidential Value of Two Social Priming Effects Through P-Curve Analyses

Abstract

It is possible that the number of false positives in the literature is much greater than is desirable due to a combination of low statistical power, publication bias, and flexibility when analyzing data. Recently, some researchers have argued the replicability crisis social priming research is greatly exaggerated (Dijksterhuis, 2014; Stroebe & Strack, 2014). To quantify the extent to which researcher degrees of freedom are a real problem, I present two p-curve analyses that examine the evidential value of research lines on professor priming and elderly priming. The results indicate studies examining elderly priming are p-hacked, while studies examining professor priming contain evidential value. I believe a polarized discussion about whether social priming is true or not, whether direct replications or conceptual replications are preferable, or whether methodological rigor or theory development is needed is unlikely to lead to scientific progress. Instead, we have to meta-analytically evaluate individual effects based on their evidential value, and collaboratively examine what is likely to be true.