Abstract
Scientometric indicators are useful to evaluate the relevance of scientific research, to prepare rankings, and to evaluate and inform research policies. That is why the choice of appropriate indicators is a matter of primary concern. This article aims to introduce a framework to decide the appropriate type of indicator for assessing the citation-based performance of complex innovation systems. The framework is two-fold: First, it brings the methodology to decide when the use of standard average based indicators is granted, and when scale-invariant indicators are mandatory. Second, it provides the procedures to build scale-invariant indicators to assess the relative impact of complex innovation systems. The framework is validated empirically through the evaluation of the relative impact of the Chilean science system in 2017. The result suggests that the Chilean science system has characteristics of a complex innovation system such as the distribution of citations fits to a power law with an exponential cutoff and a power-law correlation between the size of the system and its impact . Furthermore, the framework shows to be efficient to compare fields of vastly different sizes.
Highlights
A major feature of frequency distributions of complex innovation systems productivity is their extreme skewness (Glänzel and Nacke, 1988; Braun et al, 1990)
The research system’s output such as the number of articles it publishes, the number of patents it registers, or the number of citations it receives, follow approximately a power-law with scaling exponent α ≤ 3.0 (Lotka, 1926; de Solla-Price, 1965; Naranan, 1970; Coile, 1977; Egghe and Rousseau, 1986; Pao, 1986; Glänzel and Nacke, 1988; Naranan, 1989; Narin, 1994; OluicVukovic, 1997; Seglen, 1997; Egghe, 2005; van Raan, 2006; Milojevic, 2010; Bornmann, 2013; Brookes, 2016; Bornmann and Leydesdorff, 2017). These distributions have long tails with exponents in the range between 2 < α ≤ 3 which is the distinctive characteristic of a complex system (Katz, 1999; Katz, 2005; Katz, 2016b; Dorogovtsev and Mendes, 2000; Newman, 2005; Clauset et al, 2009; Castellani and Rajaram, 2016; Rajaram and Castellani, 2016)
This study aims to present a framework to assess the citationbased impact of research systems that are characterized by rightskewed distributions that could be described by power-law
Summary
A major feature of frequency distributions of complex innovation systems productivity is their extreme skewness (Glänzel and Nacke, 1988; Braun et al, 1990). The research system’s output such as the number of articles it publishes, the number of patents it registers, or the number of citations it receives, follow approximately a power-law with scaling exponent α ≤ 3.0 (Lotka, 1926; de Solla-Price, 1965; Naranan, 1970; Coile, 1977; Egghe and Rousseau, 1986; Pao, 1986; Glänzel and Nacke, 1988; Naranan, 1989; Narin, 1994; OluicVukovic, 1997; Seglen, 1997; Egghe, 2005; van Raan, 2006; Milojevic, 2010; Bornmann, 2013; Brookes, 2016; Bornmann and Leydesdorff, 2017) These distributions have long tails with exponents in the range between 2 < α ≤ 3 which is the distinctive characteristic of a complex system (Katz, 1999; Katz, 2005; Katz, 2016b; Dorogovtsev and Mendes, 2000; Newman, 2005; Clauset et al, 2009; Castellani and Rajaram, 2016; Rajaram and Castellani, 2016). The assessment of scientific performance within a research system has traditionally been applied through measurements of the number of documents published in peer review journals, for example, those included in the WoS or/and Scopus, by an author affiliated to an institution, a country, a field or a domain and subsequently, the number of citations these articles receive (Pan and Fortunato, 2014). Garfield (2014) persuasively stated, “Citations have become the currency of scholarship.” This idea is substantiated by international research evaluation associations quantifying research quality by using citation-based indicators
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
