Does the citation period have any effect on the informative value of selected citation indicators in research evaluations?

Marcel Clermont,Johanna Krolak,Dirk Tunger

doi:10.1007/s11192-020-03782-1

Abstract

To date, there has not been any definitive statement for bibliometric analyses from the scientific community as to how long to make the period for analysis in order to guarantee the maximum possible validity of citation analyses. Up to now, it has been particularly unclear as to whether different disciplines and citation indicators have any effect on what period for analysis to select. This is why this study aims to provide answers to these issues. Therefore, publications in the fields of business studies, biology, and medicine from 2007 to 2009 are surveyed. The citations from ten separate consecutive citation periods from 2009 up to 2018 are generated for these publications from Web of Science and selected citation indicators are calculated for them. The results show that the fundamental validity of citation indicators increases with time. However, if the goal of the analyses is to establish a ranking order, then even short citation periods are informative. Our results are stable in terms of the disciplines, although differences do occur in the citation indicators under consideration.

Highlights

There are frequent discussions in scientometric literature, as well as in higher education evaluations, about how long the period that a citation analysis is based on should be (e.g. Research Evaluation and Policy Project 2005, p. 20f.)
Long citation periods bring with them the benefit of compensating for short, random fluctuations to a certain degree by lengthening the citation period
The descriptive analyses of the three considered disciplines with regard to the ten generated citation periods for the individual indicators are shown in Tables 1, 2 and 3

Summary

Introduction

There are frequent discussions in scientometric literature, as well as in higher education evaluations, about how long the period that a citation analysis is based on should be (e.g. Research Evaluation and Policy Project 2005, p. 20f.). There are frequent discussions in scientometric literature, as well as in higher education evaluations, about how long the period that a citation analysis is based on should be Scientometrics (2021) 126:1019–1047 and evaluate the citation rate at an early stage, while looking at short citation time periods can cause the data to be distorted. With regard to investigating the research impact of German-language business schools, researchers such as Dyckhoff and Schmitz (2007) analyse comparatively long periods of 10 and 14 years, while Dilger (2010) made the citation period relatively short at two and a half years. Long citation periods bring with them the benefit of compensating for short, random fluctuations to a certain degree by lengthening the citation period. The drawback is the resulting greater effort required to survey the citation indicators

Objectives

Methods

Results