The 'Gateway Belief' Illusion: Reanalyzing the Results of a Scientific-consensus Messaging Study

Abstract

This paper analyzes data collected but not reported in the study featured in van der Linden, Leiserowitz, Feinberg, and Maibach (2015). VLFM report finding that a “scientific consensus” message “increased” experiment subjects’ “key beliefs about climate change” and “in turn” their “support for public action” to mitigate it. However, VLFM failed to compare the responses of message-exposed subjects to those of control-group subjects exposed only to “distractor” news stories unrelated to climate change. After being told that “97% of climate scientists have concluded that human-caused climate change is happening,” the former did indeed increase their own estimates of “the percentage of scientists [who] have concluded that human-caused climate change is happening.” But the degree to which they thereafter “increased” their expressed levels of belief in global warming and support for mitigation did not vary significantly, in statistical or practical terms, from the degree to which the control-group subjects increased theirs. The median and modal changes in the 101-point scales used to measure these “increases” was in fact zero for both groups. Bayesian statistical analysis confirms the data were many times more consistent with the inference that being exposed to a consensus message had no impact on either belief in climate change or support for mitigating it than with the inference that the message had any impact on these outcomes. In addition to reporting the responses of the control-group subjects, the paper shows how the absence of an experimental effect was obscured by VLFM’s use of a misspecified structural equation model. Other discrepancies between the data and VLFM’s characterizations of them, including ones relating to the impact of the experimental treatment on subjects of opposing political outlooks, are also identified.