Quantification 2.0? Bibliometric Infrastructures in Academic Evaluation

Anne K. Krüger

doi:10.17645/pag.v8i2.2575

Abstract

Due to developments recently termed as ‘audit,’ ‘evaluation,’ or ‘metric society,’ universities have become subject to ratings and rankings and researchers are evaluated according to standardized quantitative indicators such as their publication output and their personal citation scores. Yet, this development is not only based on the rise of new public management and ideas on ‘the return on public or private investment.’ It has also profited from ongoing technological developments. Due to a massive increase in digital publishing corresponding with the growing availability of related data bibliometric infrastructures for evaluating science are continuously becoming more differentiated and elaborate. They allow for new ways of using bibliometric data through various easily applicable tools. Furthermore, they also produce new quantities of data due to new possibilities in following the digital traces of scientific publications. In this article, I discuss this development as quantification 2.0. The rise of digital infrastructures for publishing, indexing, and managing scientific publications has not only made bibliometric data become a valuable source for performance assessment. It has triggered an unprecedented growth in bibliometric data production turning freely accessible data about scientific work into edited databases and producing competition for its users. The production of bibliometric data has thus become decoupled from their application. Bibliometric data have turned into a self-serving end while their providers are constantly seeking for new tools to make use of them.

Highlights

Current observations—discussed as the ‘audit’ (Power, 1999), ‘evaluation’ (Dahler-Larsen, 2012), or ‘metric society’ (Mau, 2019)—indicate a lack of societal trust in the performance of public organizations and the individuals working in them
With the rise of new public management and economic calculations about the return on public or—in the case of the US—private investment into science and Politics and Governance, 2020, Volume 8, Issue 2, Pages 58–67 higher education, universities have become subject to internal evaluations and external ratings and rankings according to their performance in research and increasingly in teaching
This has led to an increasing evaluation of the performance of individual researchers building on standardized quantitative indicators such as their publication output and their personal citation scores

Summary

Introduction

Current observations—discussed as the ‘audit’ (Power, 1999), ‘evaluation’ (Dahler-Larsen, 2012), or ‘metric society’ (Mau, 2019)—indicate a lack of societal trust in the performance of public organizations and the individuals working in them. With the rise of new public management and economic calculations about the return on public (see Schimank, 2005) or—in the case of the US (see Espeland & Sauder, 2016)—private investment into science and Politics and Governance, 2020, Volume 8, Issue 2, Pages 58–67 higher education, universities have become subject to internal evaluations (see Hillebrandt, 2020; Huber, 2020; Matthies & Simon, 2008) and external ratings and rankings (see Brankovic, Ringel, & Werron, 2018; Hazelkorn, 2011; Espeland & Sauder, 2016) according to their performance in research and increasingly in teaching (see Times Higher Education, 2018) This has led to an increasing evaluation of the performance of individual researchers building on standardized quantitative indicators such as their publication output and their personal citation scores (see de Rijcke, Wouters, Rushforth, Franssen, & Hammarfelt, 2016; Waltman, van Eck, Visser, & Wouters, 2016). Bibliometric data has turned into a selfserving end while their providers are constantly seeking for new tools to make use of them

Quantifying Science as ‘Research Performance’

The ‘Competition for Expertise’

Bibliometric Databases as Digital Infrastructures

Conclusion and Outlook