Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials

Abstract

This study aimed to assess the changes in the acceptance rates between double- and single-blind peer review systems. The search was conducted using Medline, Embase, and ClinicalTrials.gov databases as electronic databases from the inception of each database to June 2021. No restriction for language or geographic location was applied. The selection criteria included randomized controlled trials comparing the double-blind peer review process vs the single-blind peer review process. The primary outcome was manuscripts acceptance rates. The summary measures were reported as relative risk with 95% confidence intervals using the random-effects model meta-analyses. Between-study heterogeneity was explored using the I2 statistic. A total of 11 randomized controlled trials, including 3477 reviewers and 3784 manuscripts, were identified. The manuscript acceptance rates were significantly lower in the double-blind (200/1413 [14.2%]) peer review processes than in the single-blind (194/1019 [19.0%]) peer review processes (relative risk, 0.82; 95% confidence interval, 0.70-0.97; n=5 randomized controlled trials). Only 1 randomized controlled trial assessed the authors' and/or institutions' prestige on acceptance rates with results not statistically significant. Only 2 randomized controlled trials assessed the manuscript origin (US or non-US) effect on acceptance rates with results not statistically significant. Gender of the manuscript authors was assessed by only 1 randomized controlled trial, and although blinding or not female author names made no statistical difference, blinding of male author names was associated with a significant decrease in acceptance rate (139/1266 [11.0%] vs 190/1266 [15.0%]; relative risk, 0.73; 95% confidence interval, 0.59-0.90). Double-blind peer review was deemed successful by reviewers in only approximately 52% of the cases (n=5 randomized controlled trials). The double-blind peer review process seemed to be associated with an 18% lower manuscript acceptance rate than the single-blind peer review process. However, given the large heterogeneity among the included studies, more research is needed to confirm these findings and elucidate those factors that can affect the acceptance rate in double- and single-blind peer reviews.