A sensitivity analysis to methodological variation in indicator-based urban sustainability assessment: a Quebec case study

Abstract

Methodological factors are often acknowledged for their impact on the urban sustainability assessment outcomes. The significance of this impact, however, divides those who assign it great importance; to the point of compromising any attempt at sustainability benchmarking using indicators, from those who regard it as having a relatively limited role. In this paper, we aim to contribute to this debate by assessing how methodological choices could influence the variation in the outcomes of indicator-based urban sustainability benchmarking. The paper starts with a literature review outlining the evolution of the approaches adopted since the pioneering experiences in the ‘90s to show a convergence in indicators and data over time. Subsequently, by using the data of the 25 most populous cities in Quebec, we empirically analyse the sensitivity of the cities’ sustainability scores to 18 combinations of estimation techniques, and observe consistency within the outcomes while maintaining general patterns regardless of the approach used. The results suggest that methodological choices play a limited role in the urban sustainability assessment outcomes. Thus, other determinants beyond the evaluation methods are most likely involved and therefore should be a major focus in future research agendas.