Should Google Scholar be used for benchmarking against the professoriate in education?

Margaret K Merga,Sayidi Mat Roni,Shannon Mason

doi:10.1007/s11192-020-03691-3

Margaret K Merga, Sayidi Mat Roni + Show 1 more

Open Access

https://doi.org/10.1007/s11192-020-03691-3

Copy DOI

Journal: Scientometrics	Publication Date: Sep 14, 2020
Citations: 9	License type: open-access

Affiliation: Edith Cowan University, Nagasaki University

Abstract

In the neoliberal environment of contemporary academia, an individual’s research rankings and outputs can shape their career security and progression. When applying for ongoing employment and promotional opportunities, academics may benchmark their performance against that of superior colleagues to demonstrate their performance in relation to their discipline. The H-index and citation rates are commonly used to quantify the value of an academic’s work, and they can be used comparatively for benchmarking purposes. The focus of this paper is to critically consider if Google Scholar be used for benchmarking against the professoriate in education, by weighting up issues of data reliability and participation. The Google Scholar profiles of full professors at top ranked universities in Australia, the United Kingdom and the United States of America are analysed to explore how widespread Google Scholar use is in the education professoriate. Quartiles of impact are established in relation to H-index, with exploration of how gender is distributed across these quartiles. Limitations of using Google Scholar data are highlighted through a taxonomy of quality confounders, and the utility of Google Scholar as a legitimate tool for benchmarking against the professoriate in education is strongly challenged. As metrics continue to rise in their importance for academics’ job security and promotional prospects, reliance on metrics of dubious quality and uneven participation must be questioned.

Highlights

In contemporary academia, an individual’s research rankings and outputs shape their job security and progression in a highly competitive environment (Osterloh and Frey 2015)
We focus on full professors in the discipline of education as this is the ultimate goal for many academics in the discipline, and we felt that it would be useful to illustrate the commonalities and differences in this highest echelon and at highest tier institutions, in relation to the quartiles of impact we explore
We acknowledge that the path to full professorship differs between the nations we explore, and full professors in AU, the United Kingdom (UK) and the United States of America (USA) may not be considered to be exactly equivalent

Summary

Introduction

An individual’s research rankings and outputs shape their job security and progression in a highly competitive environment (Osterloh and Frey 2015). Full professorship is the ultimate goal for many academics in the discipline of education, though to secure initial employment may be a more immediate and pragmatic concern. Before the goal of full professorship can be attained, academics must secure ongoing, secure employment through tenure, and progress upward through internal promotion or mobility between institutions. Where applicants benchmark by comparing themselves against more senior colleagues within their discipline, they can show that they are meeting or exceeding disciplinary norms in research output production. While it is a common feature of such cases, little attention is given to how this benchmarking should be performed, and the implications of the activity

Methods

Results

Discussion

Conclusion