Counterfactual Simulations of Science: A Response to Farris and Revlin

Abstract

Hilary Farris and Russell Revlin derive a paradox from the literature on scientific reasoning: 'scientists conclude from their research that scientists are poor at drawing proper conclusions from their research'.' For example, they cite inconsistencies among studies investigating whether subjects and scientists can effectively employ a Popperian strategy.2 According to Farris and Revlin, we must either conclude that studies of scientific reasoning are suspect or that scientists do not behave rationally. They virtually ignore a third possibility: that these experiments demonstrate the extent to which philosophical prescriptions regarding scientific behaviour are inaccurate or incomplete. As Steve Fuller has argued, philosophical prescriptions ought to be amenable to experimental test.3 Experimental simulations using Wason's 2-4-6 task and college students are certainly not realistic, but the strength of experiments is their artificiality:4 they permit us to explore the efficacy of philosophical prescriptions under controlled circumstances. However, Farris and Revlin seem to assume that scientists ought to behave in accordance with some philosophical notion of rationality, and set out to demonstrate that behaviour which appears on the surface to be irrational is, in fact, rational. They focus on a phenomenon often labelled 'confirmation bias', and argue that it is really a manifestation of a rational strategy called 'counterfactual reasoning'. Confirmation bias was said to occur when subjects proposed positive instances of a hypothesis; for example, on Wason's 2-4-6 task,5 a subject might follow the opening '2,4,6' with '6,8,10' and '20,22,24', and fail to propose triples like '7,5,3' that would disconfirm the hypothesis 'evens ascending by twos'.