The effectiveness of journals as arbiters of scientific impact.

C E Timothy Paine,Charles W Fox

doi:10.1002/ece3.4467

C E Timothy Paine, Charles W Fox

Open Access

https://doi.org/10.1002/ece3.4467

Copy DOI

Abstract

Academic publishers purport to be arbiters of knowledge, aiming to publish studies that advance the frontiers of their research domain. Yet the effectiveness of journal editors at identifying novel and important research is generally unknown, in part because of the confidential nature of the editorial and peer review process. Using questionnaires, we evaluated the degree to which journals are effective arbiters of scientific impact on the domain of Ecology, quantified by three key criteria. First, journals discriminated against low‐impact manuscripts: The probability of rejection increased as the number of citations gained by the published paper decreased. Second, journals were more likely to publish high‐impact manuscripts (those that obtained citations in 90th percentile for their journal) than run‐of‐the‐mill manuscripts; editors were only 23% and 41% as likely to reject an eventual high‐impact paper (pre‐ versus postreview rejection) compared to a run‐of‐the‐mill paper. Third, editors did occasionally reject papers that went on to be highly cited. Error rates were low, however: Only 3.8% of rejected papers gained more citations than the median article in the journal that rejected them, and only 9.2% of rejected manuscripts went on to be high‐impact papers in the (generally lower impact factor) publishing journal. The effectiveness of scientific arbitration increased with journal prominence, although some highly prominent journals were no more effective than much less prominent ones. We conclude that the academic publishing system, founded on peer review, appropriately recognizes the significance of research contained in manuscripts, as measured by the number of citations that manuscripts obtain after publication, even though some errors are made. We therefore recommend that authors reduce publication delays by choosing journals appropriate to the significance of their research.

Highlights

Journals that peer review submissions provide the primary manner for academic researchers to disseminate their research findings (Rowland, 2002)
Of the 10,580 manuscripts about which authors answered our query, 64.8% were published in the first journal to which they were submitted, whereas those rejected at least once were submitted to a mean of 2.42 journals and took on average 478 days to be published (Figure 1). 13.0% of manuscripts took two or more years from first submission to publication, and 1.6% took four or more years, with the time to publication increasing by 234 days per rejection. 3.4% of manuscripts were rejected from four or more journals
Rejection after review was more frequent than desk rejection at most journals, but this reversed at journals with journal impact factor (JIF) greater than nine (Generalized linear mixed-effect model: p < 0.0001)

Summary

Introduction

Journals that peer review submissions provide the primary manner for academic researchers to disseminate their research findings (Rowland, 2002). Journal editors serve as gatekeepers of scholarly publishing, ensuring that research is published in the appropriate location for its quality and significance. They oversee peer review by experts, advising on editorial decisions and providing constructive feedback to improve manuscripts. Peer review gives readers confidence in the validity of the results and interpretation presented in published articles, as those articles have withstood the scrutiny of experts in the field. The peer review process is a primary mechanism by which the critical skepticism that characterizes science is put into practice (Ziman, 2000), making it the linchpin.

Methods

Results

Conclusion