The Challenges of Measuring Environmental Sustainability

Abstract

I am pleased by the publication of Whitford and Wong's article, and Social Foundations for Environmental Sustainability, in Political Research Quarterly because of the importance of the topic they address and the need for social scientists in various disciplines to empirically assess the types of hypotheses they articulate. I am well aware of the many challenges (e.g., measurement and data availability issues) researchers face in developing quantitative macro-comparative models and am sympathetic to dieir efforts. Although there are a variety of aspects of Whitford and Wong's study with which I could take issue, in many respects, I think the study is reasonably well done. However, there is one major problem that cannot be overlooked: the use of the Environmental Sustainability Index (ESI) and its components as the dependent variables in the analyses undermines the validity of the conclusions, since the ESI is a fundamentally flawed measure of sustainability. Thus, I will focus my comments here on the flaws of the ESI in the hope of improving social science research on the vital topic of environmental sustainability. The most basic problem with the ESI is that the components that make it up are combined through an effectively arbitrary weighting system and are themselves produced through combining arbitrarily weighted subcomponents that are the arbitrarily weighted combination of variables, some of which themselves are based on arbitrarily weighted indices. The result is that except for the base variables that are not indices themselves, none of which are directly modeled by Whitford and Wong, the units of the ESI, its components, and its subcomponents are largely meaningless. What, after all, would it mean if the ESI or one of its components, such as the Environmental Systems component, increased by one unit? This may in fact not indicate an improvement in sustainability, since it could reflect a variety of changes. For example, the air quality indicator could increase substantially because the concentration of sulfur dioxide in the air decreases significantly, while the biodiversity indicator decreases modestly because of an additional listing of an endangered species, with a net effect of an increase in the Environmental Systems component, and thus, the ESI. However, is it really reasonable to think that the indicators of air quality and biodiversity are of equal importance to sustainability? How does sulfur dioxide concentration compare in importance with species endangerment? It is rather hard to say, and this is exactly why I refer to the ESI and its components as arbitrarily weighted. If one slightly changed the weighting, the scenario I just mentioned (decrease in sulfur dioxide concentration and the endangerment of a species) might well lead to a decrease in the ESI instead of an increase. So, it is hard to make much of what it means to talk about improving or worsening sustainability when using the ESI. I am, of course, aware that indices of this nature are not uncommon in the social sciences, so Whitford and Wong are not deviating here from a common practice. However, the use of such arbitrarily weighted measures, particularly as dependent variables, has always been controversial, and it seems to me reasonable to avoid their use when possible. There are clearly better options than using the ESI as a sustainability measure. One can, of course, use direct measures of environmental problems or stressors, such as carbon dioxide (CO2) emissions. These have the advantage of measuring something quite real and in meaningful units. The downside is, of course, that any one such indicator is not necessarily a good indicator of sustainability in general. Another option is to use a broader measure that is based on combining different environmental impacts in a theoretically meaningful way using a logic of weighting that connects to a real unit. The ecological footprint (originally developed by Wackernagel and Rees [1996]) is just such a measure that converts a variety of human demands on the environment into a land area measure - that is, the land area necessary to support human consumption and waste production. …