Welcoming Quality in Non-Significance and Replication Work, but Moving Beyond the p-Value

Abstract

The self-correcting nature of psychological and educational science has been seriously questioned. Recent special issues of Perspectives on Psychological Science and Psychology of Aesthetics, Creativity, and the Arts have roundly condemned current organizational models of research and dissemination and have criticized the perverse incentive structure that tempts researchers into generating and publishing false positive findings. At the same time, replications are rarely attempted, allowing untruths to persist in the literature unchallenged. In this article, the editors of the Journal of Advanced Academics consider this situation and announce new policies for quantitative submissions. They are (a) an explicit call for replication studies; (b) new instructions directing reviewers to base their evaluation of a study’s merit on the quality of the research design, execution, and written description, rather than on the statistical significance of its results; and (c) an invitation to omit statistical hypothesis tests in favor of reporting effect sizes and their confidence limits.