What Makes a Good Peer Review? An Author's Perspective

Abstract

Formalised peer review began with some journals in the 18th century and has been a touchstone of modern scientific method since the middle of the 20th century. Before then, its application was lax. For instance, Albert Einstein's revolutionary “Annus Mirabilis” papers in the 1905 issue of “Annalen der Physik” were not peer reviewed (Wikipedia, No date). Later on, Einstein stopped submitting work to the journal “Physical Review” after receiving negative critique in response to a paper he had written on gravitational waves in 1936. It is possible that this was his first genuine encounter with anonymous peer review. Einstein was accustomed to gentler treatment earlier in his career and had his papers published without question or revision. Anything less must have seemed to him a tremendous slight. Einstein, who reacted angrily to the referee's report, would have been well-advised to pay more attention to its criticism, which proved to be valid (Kennefick 2005). In the early 1900s the rejection rate of journals was remarkably low – no higher than five or ten percent. I have experiences of peer review both from an author and a “peer” or “reviewer” or “scientific referee” perspective. Here I will try to keep the focus on an author's perspective. My own experience of the peer review process has changed over the years. My first experiences nearly ended my career as a researcher. My first paper that went through the peer review process was rejected and it was an enormous disappointment for me. After revision, the paper was accepted in another journal. When my second paper was also rejected in the first journal to which it was submitted, I seriously considering giving up trying to be a researcher. Thanks to my supervisor and friends, however, I got through this tough period. At the time, my colleagues seldom talked about also experiencing rejections now and then. Thus, in my world it was as if only my papers were rejected. Today a rejection does not affect me so much and there are several reasons for this. Firstly, all authors experience this setback every now and then. Second, when attending to the comments the paper definitely is improved. Third, there is evidence that the peer review process is a rather subjective one, even though it is the best that there is on offer: “This system is subjective and by no means without flaws but it is the standard for evaluating scientific work and the best system currently available to separate ‘the weeds from the flowers‘.” (Molassiotis & Richardson 2004, page 359). Some recommendations should be followed before you send your manuscript to a journal. Showing your work to others increases the probability that weaknesses will be identified and, with advice and encouragement, put right. Be careful, do not take chances due to being tired of working on your paper, and follow the guidelines of your chosen journal. Remember that it is you as an author who is responsible for the technical and scientific accuracy of your manuscript: “Finally, the day of completion comes. You know it's not quite ready, but it is time nonetheless to send it off into a black hole known only as ”the peer review process.“ (Church 1997, page 1). Waiting for the editor's decision and the validation (or not) of one's work is often associated with anxiety, especially at the beginning of one's career. During this period, it is worthwhile to read the paper through a couple of times. This gives you a chance to find errors you could not see when you were in the middle of writing and that you can correct when it is time to re-submit. If the review process is considerably delayed, your anxiety level will further increase as a publication often is connected with an author's career development, promotion or further grant applications and research work (Molassiotis & Richardson 2004). Whatever the decision, you will be in good company. In a study of the editorial process for papers submitted to a specialist rehabilitation journal, between 19% and 31% were rejected without review, 17–24% were rejected later, and the proportion accepted dropped from 64% to 47% from mid-1999 to mid-2003 (Wade & Tennant 2004). If the paper is rejected it might be difficult to not take this personally, even though the advice given is intended to be helpful and not negative. Remember that, after having considered the comments and taken actions to develop your paper, it will certainly have improved: “maintain your sense of humour throughout the publication process (or acquire one quickly if you don't have one), be persistent, and be resilient. By persistent and resilient, I mean that it's not necessary to become overly morose on receiving a rejection. Deal with it, accept it, and move on. There are plenty of publications out there. If one's written work can't find a home somewhere, it probably contains significant deficiencies.” (Kahnweiler 1997, page 7). Agreement between reviewers is often poor. Inter-reviewer agreement for Personality and Social Psychology Bulletin showed a correlation of 0.29 (Petty et al. 1999). Also in a rehabilitation journal the agreement between reviewers was low, with a coefficient varying between 0.12 and 0.27 (Wade & Tennant 2004). Thus, the importance of the editor on final decisions remains significant, particularly for papers on the margin. Read the comments carefully and carry out all the corrections indicated – they help to create a better revised paper! A rigorous review revision can be most rewarding and responsive and improve the readability of the paper. If you do not agree with a referee's comments say so politely, stating reasons for rejecting the suggestions (Molassiotis & Richardson 2004). Do not try to hide/gloss over mistakes, and correspond only with the editor. Authors and editors must be team players who are willing to listen to each other and do what is necessary to produce a clear and concise manuscript (Firestone 2004). It is important to note that the role of the referees is advisory, and the editor is under no formal obligation to accept their opinions. When an editor receives very positive and very negative reviews about the same manuscript, the editor may adjudicate between the disagreeing referees, effectively acting as the third referee: “Do not abuse or threaten a referee/editor who has rejected your paper; one day you may be in his (sic) shoes.” (Gitanjali 2001, page 213) Some recommendations for a good peer review can be made (Table 1). Personally, I prefer the referees and authors to be blinded to each other's identity. Although double-blinded peer review does not guarantee anonymity, it generates a better perception of fairness and equality in publishing. Even if double-blinded peer review is used together with explicit instructions to authors to hide the identity of the institution and/or author of a paper from the reviewers, 34% of manuscripts contained hints that “unblind” the authors (Katz et al. 2002). Referees should have the humility to refuse to scrutinise manuscripts that are outside their areas of expertise. It is also important that editors and peers work in a way that does not cause bias (for example, preventing new ideas, women, young scientists, scholars from less prestigious universities and/or from developing countries from being published) (Mainguy et al. 2005). Evaluation of a manuscript should be based on specific examples, and a statement like “not interesting” must be based on thoughtful and rational arguments rather than personal prejudices (Bernstein, No date). Objectivity is essential to ensure fairness. However, supportive comments and encouragement can generate good will in an often competitive environment. Harsh judgement and vague reviews should be avoided. Two colleagues of mine gave me two examples of such comments. The first – a harsh one – was “this manuscript is rambling and boring”. The second can be seen as an example of a humorous but vague comment, at least when taken out of context: “This manuscript contains both new and important information. Unfortunately the important information is not new and the new information is not important.” (anonymous) Finally, when writing this paper I myself received a comment from one reviewer that I want to share with you. Together with the above example, this one illustrates that humour, used with care, can also have an important place in peer-review: “It is not earth-shattering stuff, but it is the sort of paper that is useful to instrument-users and study-designers.” To produce a good peer review, peers and editors should treat their fellow researchers in a manner in which they themselves would want to be treated (Bernstein, No date). Authors should try not to take the comments and decision personally, and instead look at them objectively. They need to see the process as an opportunity to have their manuscript scrutinized, get views about how to improve the manuscript, and thus have the opportunity to produce a better paper.