The miracle of peer review and development in science: an agent-based model

Simone Righi,Károly Takács

doi:10.1007/s11192-017-2244-y

Simone Righi, Károly Takács

Open Access

https://doi.org/10.1007/s11192-017-2244-y

Copy DOI

Abstract

It is not easy to rationalize how peer review, as the current grassroots of science, can work based on voluntary contributions of reviewers. There is no rationale to write impartial and thorough evaluations. If reviewers are unmotivated to carefully select high quality contributions, there is no risk in submitting low-quality work by authors. As a result, scientists face a social dilemma: if everyone acts according to his or her own self-interest, the outcome is low scientific quality. We examine how the increased relevance of public good benefits (journal impact factor), the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information, can help the evolution of high-quality contributions from authors. High effort from the side of reviewers is problematic even if authors cooperate: reviewers are still best off by producing low-quality reviews, which does not hinder scientific development, just adds random noise and unnecessary costs to it. We show with agent-based simulations why certain self-emerged current practices, such as the increased reliance on journal metrics and the reputation bias in acceptance, work efficiently for scientific development. Our results find no proper guidelines, however, how the system of voluntary peer review with impartial and thorough evaluations could be sustainable jointly with rapid scientific development.

Highlights

Peer review is the fundamental process used by the scientific community to select and to ensure the quality of academic publications
We examine how the increased relevance of public good benefits, the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information, can help the evolution of high-quality contributions from authors
We fix j 1⁄4 2, but we show in the robustness checks reported in Fig. 15 in the ‘‘Appendix’’ that our results do not change as long as journal impact factor (JIF) is a true public good multiplier (j [ 1)

Summary

Introduction

Peer review is the fundamental process used by the scientific community to select and to ensure the quality of academic publications (cf. e.g., Alberts et al 2008; Bornmann 2013; Squazzoni and Takacs 2011). We examine how the increased relevance of public good benefits (journal impact factor), the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information, can help the evolution of high-quality contributions from authors.

Results

Conclusion