Early career researchers have questions about peer review—we asked the ASLO editors for answers

Abstract

Peer review is the formal means by which the scientific community assesses the originality, reproducibility, validity, and quality of a research study (Bakker and Traniello 2019). As such, peer review assures nonexperts that they can trust a study's findings (Jamieson et al. 2019). Despite the critical importance of peer review, graduate students, postdocs, and other early career researchers (ECRs) have limited resources for learning about this process (but see Nicholas and Gordon 2011 and Nature Communications 2021). A recent survey found that most reviewers have not received formal training on peer review and that reviewers of all career stages (77%), especially ECRs (89%), desire further training (Warne 2016). This reflects a need for guidance regarding when and how to engage in peer review, best practices for conducting a peer review, and how editors weigh peer reviews in their editorial decisions. In an effort to help new reviewers navigate this process, we (the Raelyn Cole Editorial Fellows) hosted an Association for the Sciences of Limnology and Oceanography (ASLO) webinar on peer review in September of 2021 (recording available: https://www.youtube.com/watch?v=utntl1VGy5g). The webinar had 329 registrants, including 198 students or postdocs, underscoring the desire for peer review resources. The webinar content was largely based on a survey of the associate editors (AEs) of ASLO's three peer-reviewed journals (n = 25 respondents), consisting of five open-ended questions about peer review. Here, we use insights from our survey and webinar to describe how ECRs can join the reviewer pool, provide guidance for writing a useful and time-efficient review, and discuss challenges and opportunities in the evolving landscape of peer review. Peer review begins when an AE sends invitations to potential reviewers. We asked the ASLO AEs how they select reviewers and most said they usually use one reviewer suggested by the authors. Other strategies include using the reviewer list within the ScholarOne submission website, conducting an internet search, selecting based on the AE's knowledge of experts in the field, and selecting a reviewer from the manuscript reference list. Many AEs also try to solicit diverse reviewers in terms of career stage, nationality, gender, etc. Many AEs noted the growing challenge of finding willing reviewers. Jeffrey Krause, AE for Limnology and Oceanography Letters, noted, “I typically don't have issues with finding people to request reviews from. However, getting them to accept is another challenge—my record is somewhere around 11 declines for a review for a single manuscript.” This comment reflects a trend in scholarly publishing: the proportion of reviewer invitations leading to a submitted review is declining (Fox et al. 2017; Publons 2018; Bakker and Traniello 2019). Consequently, editors must send a growing number of reviewer invitations per manuscript, ultimately increasing the time to an editorial decision. This trend is often attributed to “reviewer fatigue” (reviewers now being asked to review more manuscripts) and may reflect faster growth in article submissions than growth of the reviewer pool. One way to address reviewer fatigue/shortage is to recruit more reviewers from the underutilized ECR pool. Some AEs already intentionally solicit reviews from ECRs. “PhD students might have insights on cutting-edge topics that are lost on senior scientists,” wrote David Baker, AE for Limnology and Oceanography. Reviewing benefits ECRs: it broadens their knowledge, provides insights for improving their own manuscripts, is recognized as a form of service, and opens doors to other editorial opportunities (Schiermeier 2016; Poulson-Ellestad et al. 2020). However, some ECRs are unsure about their qualifications to review. One of the most common questions asked by webinar registrants was: “How do I know if I have the experience needed to review?” After the webinar, we sent a second survey to ASLO AEs, asking a single question: “What qualifies one to be a peer reviewer?” The most common answers indicated that reviewer selections are based on subject area expertise and publication record (Fig. 1a). Notably, holding a Ph.D. was only mentioned by three respondents, suggesting that editors welcome reviews from ECRs who have the necessary expertise and publication experience. So, how can ECRs get involved in peer review? One option is joint-peer-review, in which an ECR reviews a manuscript together with their mentor or advisor (Schiermeier 2016). All ASLO journals support this practice; invited reviewers simply notify the Editor in Chief (EIC) of their plan to conduct a joint-peer-review. This notification is critical given that confidentiality is always a top priority of peer review. Alternatively, senior scientists may provide names of ECRs as alternate reviewers when they cannot review a manuscript themselves. Finally, we encourage ECRs to contact editors of the journals for which they wish to review (via email or at a conference or other networking opportunity) and express their interest in reviewing. Any ECRs interested in reviewing for the ASLO journals may add their name to the association's ECR reviewer database (https://forms.gle/KvbmyTzrVgHKpw9x9). This database, updated periodically, is available to AEs on the dashboard from which they invite reviewers. Reviews are written both for editors and for authors: they help the former make sound editorial decisions and guide the latter on how to improve their manuscript. We asked AEs what makes a review useful, and their answers reflected these two purposes (Fig. 1b). The most common answer was that a useful review argues why a paper should be published or not, describing its strengths and weaknesses. The why argument is critical for editors and is important for both positive and negative reviews. For instance, many editors noted that they often side with the better-argued position when two reviewers disagree on their recommendation (e.g., one suggests a revision and the other suggests a rejection). This is consistent with recent work showing that AEs value the key issues identified by reviewers more than the reviewer's overall recommendation (Falkenberg and Soranno 2018). The second most common answer from AEs was that a useful review explains how the authors can improve their manuscript (Fig. 1b). “The review process should more often be about, ‘how can this manuscript be improved’ rather than ‘how can this manuscript be taken down’,” said David Antoine, AE for Limnology and Oceanography. AEs also said that useful reviews explain how the manuscript moves the field forward, describe any major issues or fatal flaws, include specific (not vague) comments, and use a polite tone. To the latter point, it may be useful to write your review, set it aside for a few days, and re-read it before submitting to ensure that your comments are appropriate and professional. Finally, it helps AEs when reviewers describe their own background and expertise. “It is a collegial approach to recognize the limits of knowledge as a reviewer,” said George Waldbusser, AE for Limnology and Oceanography: Methods. Reviewers are not expected to be experts on all aspects of the manuscript and describing the limits of your expertise can help the AE weigh the review and/or reconcile conflicting reviews. Another important aspect of peer review is time efficiency. We asked webinar participants how long they typically spend reviewing a manuscript. The most common answer was 4–8 h, but some participants answered that they spend >12 h on a review. When asked how long to spend on a review, Paul Kemp (EIC of Limnology and Oceanography: Methods) said that reviewers should invest the time and effort needed to produce the type of review they themselves would like to receive—a truly helpful review. “Everyone has their own process for reviewing,” Paul said, and one should spend whatever time it takes to produce a useful review. “But,” he added, “with practice, you get faster!” Nevertheless, there are strategies to save time when reviewing manuscripts (Table 1). We suggest that reviewers use a two- or three-read strategy, in which a quick initial read is used to identify major problems (Nicholas and Gordon 2011). If the reviewer finds a fatal flaw or a major shortcoming after their first read, they can explain the problem, including strong justification for its severity, and need not provide a full review (Table 1). It is also appropriate to contact the AE at this stage if you realize you are not well-suited to review or if the manuscript is unintelligible. The latter includes problems with language and grammar that make it difficult to understand and evaluate the scientific content. If the manuscript is intelligible and publishable in principle, you should read it a second (and optionally, third) time for detail and draft your review. In this case, you can save time by making general comments about recurring problems (Table 1). “A few examples of consistent problems is enough to make your point. It's not your job to rewrite the paper,” said Kelly Benoit-Bird, AE for Limnology and Oceanography. When commenting on grammar and linguistic precision, be aware of the disadvantages faced by non-native English-speaking authors (Clavero 2010; Ramírez-Castañeda 2020) and that being a native English speaker is not synonymous with being a good writer (Romero-Olivares 2019). The cost of professional English editing services can be a significant financial barrier for researchers from the Global South (Ramírez-Castañeda 2020). While many universities offer scientific writing assistance or graduate-student-led writing groups, assistance with English language editing often falls on favors from peers (Ramírez-Castañeda 2020). Kind and constructive feedback from a peer reviewer, such as pointing out sentences where the meaning is unclear, can be helpful; however, a reviewer's primary role is to assess scientific merit and not to fix grammatical errors. Ultimately, some researchers are calling for institutional changes to properly address linguistic injustice, such as modifying publication charges to redistribute the costs of professional English editing services so that all the financial burden does not fall on non-native English speakers (Clavero 2010; Ramírez-Castañeda 2020). Engaging more ECRs in peer review can address several current challenges in scientific publishing. ECRs can (1) expand the existing reviewer pool, (2) bring diverse perspectives to review, and (3) support positive peer review ethics. The high turnout for our webinar suggests that ECRs are interested in and motivated to participate in peer review, but many are unsure about whether they are qualified to contribute. Feedback from AEs suggests that ECRs can begin reviewing shortly after they begin to publish. We stress the importance of connecting AEs with this emerging pool of reviewers and the key role that senior scientists can play in training and advocating for ECR reviewers through joint-peer-review and suggesting ECRs as alternate reviewers. ASLO is an international society, and our peer review webinar was attended by participants from 29 countries, including 11 countries from the Global South. This global interest shows promise in diversifying the peer reviewer pool. Peer review activities are currently imbalanced across countries; for instance, researchers from the United States perform more peer reviews per article published than researchers from China (Warne 2016; Publons 2018). Furthermore, willingness to review appears higher among scholars from emerging research-output regions (such as China, Taiwan, Poland, and Brazil) than in those from established regions (such as the United States; Publons 2018). This high willingness to review persists despite language barriers for non-native English speakers, which can present challenges for authors and reviewers alike. While non-native English-speaking ECRs may be intimidated or feel unqualified to serve as peer reviewers (Ramírez-Castañeda 2020), it may be especially beneficial to their professional development to start reviewing early and often. In fact, a survey of ASLO editors showed that non-native English speakers tended to assume the role of AE after obtaining more peer review experience than their native English-speaking colleagues (Poulson-Ellestad et al. 2020). Attracting and training more reviewers from different countries can also grow the reviewer pool and bring new, diverse perspectives to reviews. Finally, we hope that efforts by ECRs to learn about the peer review process will instill positive peer review ethics. This means treating authors the way you would like to be treated (e.g., the ‘golden rule’; McPeek et al. 2009) by responding as soon as possible after receiving a review request (so as to not delay the process), reviewing 2–3 manuscripts for every paper you submit as a first author, and maintaining a professional and respectful tone in your written review. Peer review is the backbone of scientific publication; thus, constructively engaging the next generation of peer reviewers is integral to the progression of science. We thank the AEs of the ASLO journals for their thoughtful responses to our survey. We also thank Jim Cloern, Anya Metcalfe, Rita Franco-Santos, Alejandra Sanchos-Rios, and three anonymous reviewers for their helpful feedback on this editorial. Finally, we thank the ASLO EICs and our partners at Wiley, Fiona Sarne and Mia Ricci, for their help in orchestrating the webinar upon which some of this editorial is based.