Reviewer recommendations and editors’ decisions for a conservation journal: Is it just a crapshoot? And do Chinese authors get a fair shot?

Abstract

An important topic in the scientific publication process is how well reviewers evaluate the quality of papers and how their recommendations influence editors’ decisions to accept or reject papers. Additionally, a particular concern for researchers from China and other countries with rapidly developing scientific communities is whether there are potential biases affecting their manuscripts in the review process. To address these topics, we examined 4575 manuscripts submitted to the journal Biological Conservation. For the 2093 papers sent out for review, reviewer recommendations strongly influenced the outcome of the review process. Reviewer recommendations of accept and minor revision were similar in their positive effects on editor decisions, while papers receiving at least one recommendation of reject (“the kiss of death”) were almost always rejected. Papers with more consistent reviews (e.g. both reviewers recommending a major revision) had a greater chance of acceptance than did papers with more variation (e.g. minor revision and reject). We found no evidence of editor bias against papers from China; however, reviewer recommendation for papers from China had a greater degree of agreement than did reviewers of papers from English-speaking countries (e.g. intra-class correlation of 0.25 vs. 0.55), due to reviewers of papers from China often agreeing that papers should be rejected or require major revision. Reviewers from China judged papers from China more harshly than did reviewers from other countries. Our results demonstrate that the review process is not a crapshoot; reviewers are providing useful information and editors are using this information to make reasonable decisions.