Exploring the scientific impact of negative results

Abstract

Negative results are a routine part of the scientific research journey, yet they often receive insufficient attention in scientific publications. In this study, we investigate the scientific impact of negative results by comparing the citations and citation context between negative and positive results. Specifically, we compared 159 negative result papers from three journals: Journal of Negative Results in BioMedicine, PLoS One, and BMC Research Notes, with 1,058 matched positive result papers authored by the same first and corresponding authors. The citation context was categorized according to three dimensions: citation aspect, citation purpose, and citation polarity. The first two were automatically provided by Citation Opinion Retrieval and Analysis (CORA), while citation polarity was manually annotated. Our analysis revealed several key findings. Firstly, negative results received 38.6 % fewer citations than positive results, even after controlling for bibliographic factors. Secondly, negative results were associated with a significantly higher proportion of negative citations when compared to positive results. Lastly, a higher proportion of negative results were negatively cited in the methods section.