TETLOCK AND COUNTERFACTUALS: SAVING METHODOLOGICAL AMBITION FROM EMPIRICAL FINDINGS

Abstract

In five works spanning a decade, Philip E. Tetlock's interest in counterfactuals has changed. He began with an optimistic desire to make social science more rigorous by identifying best practices in the absence of non-imagined controls for experimentation. Soon, however, he adopted a more pessimistic analysis of the cognitive and psychological barriers facing experts. This shift was brought on by an awareness that experts are not rational Bayesians who continually update their theories to keep up with new information; but instead are affected by political, cognitive, and psychological heuristics, including hindsight bias, cognitive conservatism, and the fundamental attribution error. But techniques of computational simulation-involving the rigorous production of large numbers of counterfactual worlds-make it possible to mitigate both problems that Tetlock identifies: that history, produced only once, is a lousy teacher; and that humans, with their collection of non-Bayesian heuristics, are lousy pupils. Tetlock was wrong to reject this approach as theoretically promising but rhetorically and practically impractical.