Learning from Biased Research Designs

Abstract

Most contemporary empirical work in political science aims to learn about causal effects from research designs that may be subject to bias. We provide a Bayesian framework for understanding how researchers should approach the general problem of inferring causal effects from potentially biased research designs. Our core contention is that any sincere claim that a research design contains bias entails a belief about what that bias is. Once this belief is specified (along with a prior belief about the causal effect), what one should learn from a potentially biased estimate can be derived from Bayes’s rule. We apply this principle to explore when we should learn more or less from basic difference of means estimates and then extend our analysis to speak to several methodological debates and practical problems confronting applied researchers.