Validating Vignette Designs with Real-world Data: A Study of Legal Mobilization in Response to Land Grievances in Rural China

Abstract

AbstractHow well do vignette designs capture actual behaviour in the real world? This study employs original survey data featuring both hypothetical vignettes and behavioural questions in order to assess the external validity of descriptive and causal inferences in survey experiments. The survey was conducted in a three-province, probability-proportional-to-size sample of 1,897 rural residents in China and focuses on the legal mobilization of citizens in response to grievances involving land rights. In terms of descriptive inference, we find that relative to the behavioural benchmark, hypothetical vignettes significantly over-estimate legal mobilization in response to a grievance, particularly for higher-cost actions like petitioning the government and litigating in court. We find that data from hypothetical vignettes affect causal inference as well, producing significantly different results regarding the effect of political connections and legal knowledge on legal mobilization. The study makes a contribution by identifying conditions under which hypothetical vignettes are less likely to produce valid inference. It engages a rich literature on disputing and legal mobilization in the field of Chinese politics and helps to resolve debates over the role of political connections and legal knowledge.