Peer Review in a Small Pond

Abstract

In the world of academic publishing, the peer review process carries heavy freight, serving as the prime instrument for quality control, adjudication of importance, quality improvement and safeguarding academic integrity. These high stakes underline the need for fair and unbiased assessment – a special challenge for Healthcare Policy because of its emphasis on the Canadian context and the still small pool of Canadian health services and policy researchers. Many of these researchers are well known to one another as colleagues, collaborators, competitors, friends and foes. And as health services and policy researchers increasingly engage with decision-makers – integral players in the journal’s review process – the web of relationships becomes even more complex. Under these circumstances, how does the journal attempt to ensure a fair process? The editor-in-chief or a senior editor assesses each submitted manuscript with authors’ names removed to determine whether it will undergo formal peer review. When an editor nevertheless recognizes the author(s) and feels uncertain about his or her ability to make an unbiased decision, the editor removes him- or herself from the review process and another editor takes over. Rejection without peer review is based either on content that lies outside the journal’s declared scope or reflects major reservations about study design or execution. Any manuscript that is being considered for rejection without peer review is further assessed by a second editor before a decision is made. Having cleared the initial hurdle, the manuscript is assigned to an editor who shepherds it through the peer review process, assigning academic and decision-maker reviewers, reviewing their assessments and drafting a letter to the authors. A minimum of two academic reviews and one decision-maker review are required. The reviewers are blinded to the identity of the authors, although there are undoubtedly times when they recognize or suspect who they are. Reviewers are advised that they “must disclose … any conflicts of interest that could bias their opinions of the manuscript” and “disqualify themselves … if such a conflict exists.” The editor-in-chief vets the reviews and the draft letter to the author, sometimes adding comments, suggestions or requests. Throughout this process, we emphasize the quality-improvement dimension of peer review, encouraging our reviewers (and editors) to be what Taylor (2003) describes as “big R” reviewers – respectful, constructive and considerate. Reviewers are requested to provide “constructive comments and suggestions,” to “include strengths and limitations” and to “indicate specific areas that might be improved.” In our editors’ letters, we try to assist the authors to strengthen both the substance and the presentation of their research or commentary – to help them produce a publishable manuscript. Ultimately, the success of these efforts depends on the ability, integrity and generosity of our editors and reviewers. So far, we think, so good. “Thank you so much … to the reviewers for such thorough and insightful comments.” “Please thank [two of the journal’s editors] for their letter and comments they have provided. … I found the comments extremely helpful. …” “I found the quality of the reviewers’ comments … to be extremely good. They had great insight into the issues, had clearly read the paper closely and made some excellent suggestions. Thanks to these reviewers, I believe the manuscript is much improved.”