Metrics and peer review agreement at the institutional level

Abstract

Several countries partly fund universities on the basis of scientific performance. In this context, peer review is often used to assess the quality of scientific products. In large research assessment exercises peer review may become costly, and bibliometric indicators have been suggested as an alternative. A recurrent question is whether peer review and metrics tend to yield similar outcomes. In this paper, we study the agreement between bibliometric indicators and peer review at the institutional level. Additionally, we also quantify the internal agreement between two peer reviewers. We find that the level of agreement is generally higher at the institutional level than at the publication level. Overall, the agreement between metrics and peer review is on par with the internal agreement among two peer reviewers. This suggests that bibliometric indicators may indeed be considered as an alternative to peer review for large institutional research assessment exercises.