Transparency Of Peer Review Research Articles

The Association between Four Citation Metrics and Peer Rankings of Research Influence of Australian Researchers in Six Fields of Public Health

Doubt about the relevance, appropriateness and transparency of peer review has promoted the use of citation metrics as a viable adjunct or alternative in the assessment of research impact. It is also commonly acknowledged that research metrics will not replace peer review unless they are shown to correspond with the assessment of peers. This paper evaluates the relationship between researchers' influence as evaluated by their peers and various citation metrics representing different aspects of research output in 6 fields of public health in Australia. For four fields, the results showed a modest positive correlation between different research metrics and peer assessments of research influence. However, for two fields, tobacco and injury, negative or no correlations were found. This suggests a peer understanding of research influence within these fields differed from visibility in the mainstream, peer-reviewed scientific literature. This research therefore recommends the use of both peer review and metrics in a combined approach in assessing research influence. Future research evaluation frameworks intent on incorporating metrics should first analyse each field closely to determine what measures of research influence are valued highly by members of that research community. This will aid the development of comprehensive and relevant frameworks with which to fairly and transparently distribute research funds or approve promotion applications.

Transparency in peer review

Would the publication of anonymous referee reports and editorial decision letters of published papers benefit the scientific debate? Results from a trial seem to suggest this.

Naming peer reviewers in JRSM

The received wisdom is that peer review is a thankless task. Peer reviewers are overworked, unpaid, unacknowledged and generally unrewarded. Peer-review processes are not standardized, and each journal or grant review body has its way of deciding what should be approved for publication or funding, respectively. Peer review has also been a secretive process that was largely unstudied until the 1990s. The evidence on peer review in journals has improved in that time, although the central conclusion still holds: peer review is best for improving the quality of articles that editors decide to publish but it is not an objective decision-making process for selecting the best articles. A publication in a major journal is no guarantee of quality and will probably contain several errors, since peer review is poor at eradicating those errors or detecting research misconduct. Sensible journal editors have accepted this reality, and acknowledge the flaws in the system they supervise. Beyond this honest appraisal of peer review, journals can help by introducing open peer-reviewer processes whereby the identity of authors and peer reviewers are not hidden to each other. The research evidence suggests that open peer review does not affect the quality of decision-making at a journal but improves the quality of the conversation between the authors, reviewers and journals. The JRSM has an open peer-review process. The identity of peer reviewers, however, is still usually hidden from readers. Sometimes authors are kind enough to thank peer reviewers in their acknowledgements at the end of a paper. Some journals list peer reviewers without linking them to specific papers – a policy that the JRSM used to follow. A few other journals pay reviewers a small amount or offer them institutional or membership discounts, an existing JRSM benefit. Nonetheless, the reward that peer reviewers receive is paltry in relation to their effort and the contribution that a good peer reviewer can make to improve a paper. Many peer reviewers justify the effort as a necessary contribution to scientific discourse but also as a contribution to their personal development. Many are simply interested enough in the topic to lend their services. Scientific journals only thrive because of the passion of peer reviewers and the patience of readers. From 2010, the JRSM will be taking a bold step in improving the transparency of peer review. We will name peer reviewers on each peer-reviewed article that we publish. This journal does not have the resources to conduct a randomized trial before making this change as bigger journals are able to do. But the editorial team believes that it is the right decision in terms of openness. Any peer reviewers who do not wish to be named can simply decline to peer review the article they have been sent. We would welcome your views on this small but important development.

An Author's Guide to Publication Ethics: A Review of Emerging Standards in Biomedical Journals

Universal definitions of ethical issues related to medical publishing have remained somewhat elusive. Training in the art of writing for medical journals is inconsistent and most commonly informal, involving collaborative efforts between mentors and students. This approach inadvertently may perpetuate erroneous assumptions as to what constitutes acceptable behavior. In contrast to instruction on composition, ethical considerations related to the publication of a paper are likely to receive little attention. Even so, consequent to the ever-increasing scrutiny from the media and government agencies, journals are recognizing the need for greater transparency in peer review and are thus more inclined to enforce ethical standards. Understanding that some apparent ethical contraventions are the result of confusion or a lack of knowledge, some journals are assuming the responsibility of educating their community about ethical issues in publishing. This paper reviews the key ethical issues (eg, authorship criteria, conflicts of interest, redundant publication, data access and biases in data reporting, image manipulation) that authors should consider before submitting a manuscript. It also surveys some of the policies of the most highly cited clinical medical journals. In the future, authors can anticipate that their submissions will be required to meet an expanding array of ethical standards.

Food-poisoning at sea.

<h3>Abstract</h3> <h3>Purpose</h3> Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer review process. Peer review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. <h3>Methods</h3> Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended “reject,” we coded for alignment between reviewers’ comments to the editor and to authors. We report descriptive statistics. <h3>Results</h3> 49% of reviews contained comments to the editor (n=176). Most of these comments summarized the reviewers’ impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 48 reviews recommending “reject,” the majority of comments to the editor contained content that also appeared in comments to the authors (65%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (85%). <h3>Conclusion</h3> Findings indicate variability in how reviewers use the confidential comments to editor section in online peer review systems, though generally the way they use them suggests integrity and transparency to authors.