Reviewing and Revamping the Double-Blind Peer Review Process

Abstract

The traditional double-blind peer review process currently used to determine which articles are published in scientific journals is far from perfect. This article argues that the Internet can provide us with a better way to judge article quality using the opinion of every reader rather than that of only a couple of reviewers. The article offers a relatively simple business model that can provide funding for such a publishing system. The model contains three basic components, a reviewing component, a submission/cost component, and a distribution component, The reviewing component will be an electronic market through which quality feedback can be bought and sold. The submission/cost component will decide if an article should be published and distributed electronically on-line using volume forecasts by experienced forecasters who will be compensated according to the accuracy of their forecasts. Articles can be bought by anyone in the scientific or general community from the distribution component. Although not perfect, our proposal does possess many features that could be valuable to the scientific community. Problems with the current peer review system It is easy to see that a reviewer’s evaluation of an article’s quality will be biased due to conflict of interest if authors and reviewers know each other. To mitigate this conflict about 350 years ago scientists introduced the double-blind peer review process – neither the author(s) nor the reviewer(s) of an article have access to each other’s identity. However when the article reviewed supports the reviewer’s prior research its evaluation will be higher on average and vice versa, so the conflict of interest is still not completely eliminated. Moreover, Statistics tells us that the personal opinions of several randomly selected reviewers cannot be a very good predictor of the value of the article to the whole scientific community (see The Economist, 1996 and Starbuck, 2004). It becomes clear that to improve the review process we need to pick reviewers whose opinions are forwardlooking and highly representative of those held by the general community. We also need to compensate somehow quick, accurate and constructive reviewers. Currently the complex task of picking and rewarding reviewers is assigned to the journal editors. Since rewards are granted by the editor, reviewers have an incentive to provide a review that matches the editor’s expectations. This means that a good review requires that the editor be also forward-looking and highly representative of the general community. Reviewers, editors, and the scientific community as a whole face a challenging task in this regard. Scientific rigor requires time. The judgment of whether a certain theory is right or wrong is not easy. It often takes years to establish the validity of a newly proposed theory. To solve the problem of quality uncertainty most universities try to estimate the prestige of the journal in which an article appears as a rough measure of an article’s quality. The prestige of a