Questionable Metascience Practices

Abstract

Questionable research practices may reduce the public’s trust in science. The present article considers some questionable metascience practices (QMPs) that may threaten scientists’ trust in metascience. A QMP is a research practice, assumption, or perspective that has been questioned by several commentators as being potentially problematic for the credibility of metascience and/or the science reform movement. The present article reviews 10 QMPs that relate to criticism, replication, bias, generalization, and the characterization of science. Specifically, the following QMPs are considered: (1) rejecting or ignoring self-criticism; (2) a fast ‘n’ bropen scientific criticism style; (3) overplaying the role of replication in science; (4) assuming a replication rate is “too low” without specifying an “acceptable” rate; (5) an unacknowledged metabias towards explaining the replication crisis in terms of researcher bias; (6) assuming that researcher bias can be reduced; (7) devaluing exploratory results as more “tentative” than confirmatory results; (8) presuming that QRPs are problematic research practices; (9) focusing on knowledge accumulation as an index of scientific progress; and (10) focusing on specific scientific methods. It is stressed that only some metascientists engage in some QMPs some of the time, and that these QMPs may not always be problematic. Research is required to estimate the prevalence and impact of QMPs. In the meantime, QMPs should be viewed as invitations to ask “questions” about how we go about doing metascience rather than as grounds for mistrusting the credibility of metascience.