The Credibility Chasm in Policy Research from Academics, Think Tanks, and Advocacy Organizations

Abstract

How do key policy professionals inside government view various sources of policy research? Are there systematic differences in the perceptions of the quality and credibility of research derived from different sources? This is a replication of and expansion on Doberstein (2017), which presented a randomized controlled survey experiment using policy analysts to systematically test the source effects of policy research. Doberstein's experimental findings provide evidence for the hypothesis that academic research is perceived to be substantially more credible to government policy analysts than think tank or advocacy organization research, regardless of its content, and that sources perceived as more ideological are much less credible. This study replicates that experiment in three additional Canadian provincial governments to verify whether the relationship found in the original study persists in a larger sample and in conjunction with further randomization procedures. This study corroborates the original study's findings, confirming that external policy advice systems are subject to powerful heuristics that bureaucrats use to sift through evidence and advice.