Review and new evidence on composite innovation indicators for evaluating national performance

Abstract

The purpose of this contribution is to present a survey of the recent developments in constructing composite science and technology (S&T) indicators on a national level as well as new evidence of the variability of such S&T indicators which opens the gateway to “country-tuning”. It has become standard practice to combine several indicators for science, technology, and innovation to form composite numbers. Especially in the light of this variability, two questions arise. Firstly, are the results (especially rankings) stable with respect to weights? Secondly, is there hope to define “economically” reasonable weights? In order to provide answers to these questions, we use data from the European Innovation Scoreboard 2005 (EIS 2005) to exemplify our reasoning. Concerning the first question, we give genuine evidence on the existence of immense variability, possibly invalidating the results. Further, we also show that even existing and well-accepted methods, like equal weighting, Benefit of the Doubt weighting (BoD) and principal component analysis weighting (PCA) may lead to drastically differing results. Concerning the second question we will demonstrate that by each composite indicator weighting a set of shadow prices is implied expressing one indicator in terms of another. Whether the weights are sensible should be evaluated on the basis of these shadow prices. It turns out that those implied by EIS 2005 contain strange peculiarities. After that we plead for more care in constructing composite indicators. Especially weights should be chosen on the basis of shadow prices, rather than, say, by equal weighting or other automatic methods. Lastly, we discuss the merit of composite indicators and argue that they have a valuable communication and competition function, but they should be accompanied by multidimensional representations, which provide the basis for the construction of policy measures.