Are complex causal models less likely to be true than simple ones? A critical comment on Trafimow (2017).

Abstract

Trafimow (2017) used probabilistic reasoning to argue that more complex causal models are less likely to be true than simpler ones, and that researchers should be skeptical of causal models involving more than a handful of variables (or even a single correlation coefficient) [Trafimow, D. (2017). The probability of simple versus complex causal models in causal analyses. Behavior Research Methods, 49, 739-746]. In this comment, I point out that Trafimow's argument is misleading, and reduces to the observation that more informative models (that make definite statements about certain causal relations) are less likely to be true than less informative models (that remain silent about those relations, by omitting some variables from consideration). This correct but trivial statement does not deliver the epistemological leverage promised in the paper. When complexity is evaluated with reasonable criteria (such as the number of nonzero effects in alternative models involving the same variables), more complex models can be more, less, or equally likely to be true compared with simpler ones. I also discuss Trafimow's claim that, if a model is unlikely to be true a priori, researchers will seldom be able to gather evidence of sufficient quality to support it; in practice, even low-probability models can receive strong support without the need for extraordinary evidence. Researchers should evaluate the plausibility of causal models on a case-by-case basis, and be skeptical of overblown claims about the dangers of complex theories.