Significance chasing in research practice: causes, consequences and possible solutions.

Abstract

The low reproducibility of findings within the scientific literature is a growing concern. This may be due to many findings being false positives which, in turn, can misdirect research effort and waste money. We review factors that may contribute to poor study reproducibility and an excess of 'significant' findings within the published literature. Specifically, we consider the influence of current incentive structures and the impact of these on research practices. The prevalence of false positives within the literature may be attributable to a number of questionable research practices, ranging from the relatively innocent and minor (e.g. unplanned post-hoc tests) to the calculated and serious (e.g. fabrication of data). These practices may be driven by current incentive structures (e.g. pressure to publish), alongside the preferential emphasis placed by journals on novelty over veracity. There are a number of potential solutions to poor reproducibility, such as new publishing formats that emphasize the research question and study design, rather than the results obtained. This has the potential to minimize significance chasing and non-publication of null findings. Significance chasing, questionable research practices and poor study reproducibility are the unfortunate consequence of a 'publish or perish' culture and a preference among journals for novel findings. It is likely that top-down change implemented by those with the ability to modify current incentive structure (e.g. funders and journals) will be required to address problems of poor reproducibility.