Abstract
BackgroundHigh predictive validity – that is, a strong association between the outcome of peer review (usually, reviewers' ratings) and the scientific quality of a manuscript submitted to a journal (measured as citations of the later published paper) – does not as a rule suffice to demonstrate the usefulness of peer review for the selection of manuscripts. To assess usefulness, it is important to include in addition the base rate (proportion of submissions that are fundamentally suitable for publication) and the selection rate (the proportion of submissions accepted).Methodology/Principal FindingsTaking the example of the high-impact journal Angewandte Chemie International Edition (AC-IE), we present a general approach for determining the usefulness of peer reviews for the selection of manuscripts for publication. The results of our study show that peer review is useful: 78% of the submissions accepted by AC-IE are correctly accepted for publication when the editor's decision is based on one review, 69% of the submissions are correctly accepted for publication when the editor's decision is based on two reviews, and 65% of the submissions are correctly accepted for publication when the editor's decision is based on three reviews.Conclusions/SignificanceThe paper points out through what changes in the selection rate, base rate or validity coefficient a higher success rate (utility) in the AC-IE selection process could be achieved.
Highlights
Reputable scientific journals only publish manuscripts that have been subjected to peer review – that is, critical scrutiny by scientific experts
We have examined in three publications [3,4,5] the predictive validity of the selection decisions at the journal Angewandte Chemie International Edition (AC-IE)
If the usefulness of peer reviewing for the manuscript selection process is determined using the model developed by Taylor and Russell [13], it must first be established what criterion will be used for considering a submission suitable for publication
Summary
Reputable scientific journals only publish manuscripts that have been subjected to peer review – that is, critical scrutiny by scientific experts. On the basis of these recommendations, the journal editor makes the decision to accept or reject for publication. This means that experts in a research area are consulted for the reviewing of the manuscript, the reviewing does not include the selection decision on the submissions: Peer review forms the basis for the selection decision made by an editor. High predictive validity – that is, a strong association between the outcome of peer review (usually, reviewers’ ratings) and the scientific quality of a manuscript submitted to a journal (measured as citations of the later published paper) – does not as a rule suffice to demonstrate the usefulness of peer review for the selection of manuscripts. It is important to include in addition the base rate (proportion of submissions that are fundamentally suitable for publication) and the selection rate (the proportion of submissions accepted)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
