A critical scientific evaluation of a purportedly negative data report - response to Seneviratne et al. 2022.

Abstract

A core principle in the pursuit of scientific knowledge is that science is self-correcting and that important results should be replicable. Hypotheses need to be reinforced, adjusted, or rejected when novel results are obtained. Replication of results confirms hypotheses and enhances their integration into scientific practice. In contrast, publication of substantiated and replicated negative findings (i.e., non-significant or opposite findings) can be the basis to reject erroneous hypotheses or develop alternative strategies for investigation. Replication is a problem in all research fields. The Psychology Reproductivity Project reported that only 36% of 'highly influential' published research in highly ranked journals were reproduced. Similar to positive data, negative data can be flawed. Errors in a negative data set can be based on methodology, statistics, conceptual defects, and flawed peer review. The peer review process has received progressive scrutiny. A large-scale review of the peer review process of manuscripts submitted to the British Medical Journal group indicated that the process could be characterized as inconsistent, inaccurate, and biased. Further analysis indicated that the peer process is easily manipulated, indicative of a failed system, is a major factor behind the lack of replication in science (acceptance of flawed manuscripts), suppresses opposing scientific evidence and views, and causes gaps in and lack of growth of science. Complicating the integrity of scientific publication is the role of Editors/Researchers. Ethical guidelines exist for major publishing houses about editorial ethics, behavior, and practice.