Officers King and Zeng and the Case of the Unsupported Counterfactual

Abstract

Let me begin by praising Gary King and Langche Zeng for their service to the field with the methods they describe in their paper (King and Zeng 2006a). It is important to know when counterfactual statements drawn from statistical estimates wander far from the data used in the estimates. I particularly like the subtlety of the observation that even when all the values of the individual variables occur in the data, it may not be the case that all combinations of those values do. There are countries that protect the civil liberties of their citizens and those that abuse them and there are democracies and autocracies, but there are not very many autocracies that protect civil liberties or democracies that abuse them. Consequently, our inferences about autocracies with civil liberties and democracies without them are not based on the same depth of data that our inferences about democracies with civil liberties and autocracies without them are. I am looking forward to using their methods to examine some of my own recent data analysis on compliance with the laws of war.' For example, U.S. military dominance characterized the Kosovo bombing and the Iraq War, but how different were these cases from the record of earlier twentieth century wars? Like Shakespeare's Marc Anthony, however, one does not get asked to contribute to such fora to praise Caesar but to bury him. While I have neither the desire, the ability, nor the intent to bury Professors King and Zeng, I do have a clod or two of dirt to toss in their general direction. My qualifications to their excellent and important article concern theory as the source of counterfactual conclusions and the range of evidence we can use to judge among competing theories. Their paper seems to assume that theory is fully captured by a well-specified regression equation, including functional form of the relationship, as judged by its ability to represent a given data set. It is as if they draw their epistemological inspiration from two fictional police officers: Captain Louis Renault of the movie Casablanca, Round up the usual suspects!, and Sergeant Joe Friday from the TV series Dragnet, 'Just the facts, ma'am. At times, I get the impression that theory plays no role in their process beyond providing a list of exogenous variables-the usual suspects-along with the appropriate specification of functional form and link function between those variables and the dependent variable, which is then applied to just the facts. But theories are more than just a list of relevant variables and a functional form; they are logical structures that provide arguments about why the variables cause one another. There are other forms of investigation than those of Captain Renault and Sergeant Friday. Theory is critical for counterfactuals because theories explain why the counterfactual should hold. Reasons why the counterfactual should be true are as