Multiple Studies and Evidential Defeat

Abstract

You read a study in a reputable medical journal in which the researchers report a statistically significant correlation between peanut butter consumption and low cholesterol. Under ordinary circumstances, most of us would take this study to be at least some evidence that there is a real causal connection of some sort (hereafter: a real connection) between peanut butter consumption and low cholesterol. Later, you discover that this particular study isn’t the only study that the researchers conducted investigating a possible real connection between peanut butter consumption and some health characteristic. In fact, their research is being funded by the Peanut Growers of America, whose explicit goal is to find a statistical correlation between peanut butter consumption and some beneficial health characteristic or other that they can highlight in their advertising. Though the researchers would never falsify data or do anything straightforwardly scientifically unethical,1 they have conducted one thousand studies attempting to find statistically significant correlations between peanut butter consumption and one thousand different beneficial health characteristics including low incidence of heart disease, stroke, skin cancer, broken bones, acne, gingivitis, chronic fatigue syndrome, carpal tunnel syndrome, poor self-esteem, and so on. When you find out about the existence of these other studies, should that at least partially undermine your confidence that there is a real connection between peanut butter consumption and low cholesterol? In other words, does the existence of the other studies at least partially defeat the evidence that the study provides for a real connection between peanut butter consumption and low cholesterol? That is the question that this paper will address. Call a defender of an answer of “yes” to this kind of question a Defeatist, and call a defender of the opposite view an anti-Defeatist. A few clarifications: First, there is the familiar point that A correlating with B isn’t the same as A causing B, so it isn’t obvious that the original study ever gives us any reason to think that eating peanut butter causes people to have low cholesterol; perhaps having low cholesterol causes people to eat peanut butter, or perhaps some other characteristic (like being tall) causes people both to eat peanut butter and to have low cholesterol.2 So let’s just focus on the question of whether there is some real causal connection or other3 between peanut butter consumption and low cholesterol. It’s very plausible that the study is evidence that there is such a real connection, whatever its nature might be.