Does uncertainty in single indicators affect the reliability of composite indexes? An application to the measurement of environmental performances of Italian regions

Abstract

In recent decades, the measurement and evaluation of important social and natural phenomena has significantly evolved, with many traditional measurements based on single variables increasingly being replaced by multidimensional approaches. One key aspect of these approaches is the development of composite indexes, usually real-value functions of multiple achievements of a group of units. The achievements in each of the selected dimensions are generally synthesised through one or more variables, often referred to as indicators. When indicators are obtained through an estimation process, it is crucial to understand if and how their estimation error – for example, sampling error – affects the resulting composite index.This paper presents a methodology based on a parametric bootstrap technique that evaluates to what extent uncertainty in indicators affects the reliability of the aggregate composite index. The method is applied to four composite indexes measuring the environmental performances of Italian regions based on real population and survey data.To our knowledge, this is the first attempt to measure the impact of indicators’ sampling error on composite indexes. If adequately generalised, our methodology could be used in the presence of measurement errors, non-response issues, or other kinds of non-sampling errors.