A Flexible Inventory of Survey Items for Environmental Concepts Generated via Special Attention to Content Validity and Item Response Theory

Abstract

We demonstrate how many important measures of belief about the environmental suffer from poor content validity and inadequate conceptual breadth (dimensionality). We used scholarship in environmental science and philosophy to propose a list of 13 environmental concepts that can be held as beliefs. After precisely articulating the concepts, we developed 85 trial survey items that emphasized content validity for each concept. The concepts’ breadth and the items’ content validity were aided by scrutiny from 17 knowledgeable critics. We administered the trial items to 449 residents of the United States and used item response theory to reduce the 85 trial items to smaller sets of items for use when survey brevity is required. The reduced sets offered good predictive ability for two environmental attitudes (R2 = 0.42 and 0.46) and indices of pro-environmental behavior (PEB, R2 = 0.23) and behavioral intention (R2 = 0.25). The predictive results were highly interpretable, owing to their robust content validity. For example, PEB was predicted by the degree to which one believes nature to be sacred, but not by the degree of one’s non-anthropocentrism. Concepts with the greatest overall predictive ability were Sacredness and Hope. Belief in non-anthropocentrism had little predictive ability for all four response variables—a claim that previously could not have been made given the widespread poverty of content validity for items representing non-anthropocentrism in existing instruments. The approach described here is especially amenable to incremental improvement, as other researchers propose more informative survey items and potentially important concepts of environmental beliefs we overlooked.