The effectiveness of peer review in identifying issues leading to retractions

Abstract

Retractions are necessary to remove flawed research from citable literature but cannot offset the negative impact those publications have on science advances and public trust. The editorial peer-review process is intended to prevent flawed research from being published. However, there is limited empirical evidence of its effectiveness in identifying issues that lead to retractions. This study analyzed the peer-review comments (provided by Clarivate Analytics) for a sample of retracted publications (provided by Retraction Watch) to investigate how the peer-review process effectively detects the areas where the retraction causes lie and whether reviewer characteristics are related to the effectiveness. We found that a small proportion of peer reviews suggested rejections during the peer review stage, while about half suggested acceptance or minor revision for those later retracted papers. The peer-review process was more effective in identifying retraction causes related to data, methods, and results than those related to text plagiarism and references. Additionally, factors such as reviewer seniority and the level of match between reviewers’ expertise and the submission were significant in determining the possibility of peer reviews identifying suspicious areas in submissions. We discussed potential insights from these findings and called for collective efforts to prevent retractions.