Assessing a New Measure of State Policy Mood: Response to Lagodny, Jones, Koch, and Enns

Abstract

AbstractThis article presents a short summary of the conclusions we report in a longer manuscript (available in our Supplementary Material) subjecting Lagodny et al.’s new measure of state policy mood to the same set of face validity and construct validity tests we applied earlier to Enns and Koch’s measure. We encourage readers to read this longer manuscript, which contains not only the conclusions herein, but also the evidence justifying these conclusions, before accepting or rejecting any claims we make. Our results show that the characteristics of Enns and Koch’s measure that led us to be doubtful that it is valid are also present in Lagodny et al.’s new measure – leaving us just as doubtful that Lagodny et al.’s measure is valid. Moreover, the low correlation between Lagodny et al.’s measure and Enns and Koch’s measure, combined with evidence from replications of seven published studies that the two measures frequently yield quite different inferences about the impact of policy mood on public policy, indicate that Lagodny et al.’s claim that both their measure and Enns and Koch’s measure are valid is wrong; either neither measure is valid, or one is valid and the other is not. Moreover, extending the replications to include not only Lagodny et al.’s and Enns and Koch’s measures, but also Berry et al.’s measure and Caughey and Warshaw’s measure of mass economic liberalism, shows that each of the four measures yields a substantive conclusion about the effect of policy mood that is dramatically different than each of the other three measures. This suggests that the goal of developing a measure of state policy mood that would be widely accepted as valid remains elusive.