Assessing the Validity of Different Approaches to Operationalizing Cultural Theory in Survey Research

Abstract

Cultural Theory (CT) is a social and political theory developed by Emile Durkheim, Mary Douglas, Aaron Wildavsky, and others that has attracted significant attention across the social sciences. This paper explains what CT is and compares the validity of various approaches to operationalizing CT in survey research. Approaches assessed for their face and content validity include (1) the worldview measures originally developed by Karl Dake and Wildavsky, (2) the relational ranking and rating measures recently created by Hank Jenkins-Smith and colleagues, (3) the relational “cultural cognition” measures of Dan Kahan and colleagues, and (4) the brand new relational grid-group measures developed by Joe Ripberger, Brendon Swedlow, and colleagues. The convergent validity of the last three relational approaches is assessed by the extent to which these measures converge on the original Dake and Wildavsky worldview measures, while their predictive validity is assessed by how well they predict people’s concerns in nine policy areas. We find that all of these approaches lack face and content validity because they at best only operationalize a component of CT. But the worldview measures are particularly lacking because worldviews do not correspond to any component of CT. By contrast, the new relational measures exhibit very good face and content validity with respect to the relational patterns hypothesized by CT. The new relational rating measures developed by Jenkins-Smith and colleagues also exhibit very good convergent and predictive validity, but the comparable results for the new grid-group relational measures developed by Ripberger, Swedlow, and others are at best mixed, yet not without promise if developed further. [The convergent and predictive validity of the relational measures of Kahan and colleagues remains to be assessed.] Based on these findings, we suggest appropriate uses for these measures and directions for further research to improve the conceptualization and operationalization of CT in survey research.