Introduction: Rational Data Choice

Abstract

We live in an era of unprecedented abundance of cross-national political data. The study of comparative politics disposes of more quantitative information than anytime before, and year by year number of cross-national datasets keeps expanding. Even if we still lack data for innumerable research questions, many areas of inquiry host not just one, but several competing datasets. We thus face luxury, and necessity, of choice-a choice, however, for which we are not well equipped as a discipline. Since data production in comparative politics essentially proceeds in a private, decentralized, and unregulated fashion (see Schedler 2012), users of cross-national political data face structural problems of information about data supply and data quality. To make rational, that is, informed and justified choices among competing datasets, they must overcome these informational uncertainties. Which is easier said than done. Over past years, practitioners of comparative politics have been paying increasing attention to issues of cross-national measurement.1 Yet use of problematic data sets (Herrera and Kapur 2007, 372) continues to be widespread, and so is uncritical choice among competing datasets. Irrational, that is, blind and nonjustified data choices are disciplinary failures as much as they are individual ones. Comparative political science has not yet developed requisite infrastructure, norms, and practices that would allow scholars to choose among available data in reflexive and reasoned ways, without previously having to make huge investments in acquisition of basic consumer information. Selecting data consciously, rather than faithfully, involves four seemingly simple tasks whose realization still imposes high information costs on data users in many fields of comparative research: (1) surveying supply of relevant data, (2) assessing quality of available data, (3) estimating inferential implications of alternative datasets, and (4) choosing appropriate data for empirical research. Task 1: Surveying Data Supply Buyers in consumer markets do not need perfect market information. They can rely on brands, habits, or emotional appeals. Critical data users must not. Before choosing their quantitative inputs, they need to inform themselves broadly about existing supply of data. What is out there? Which datasets exist in a given field of research? What do they pretend to measure? Do they offer original or composite data? Factual measures, subjective data, or expert judgments? Which are their sources? Which are units of analysis? How many countries and which period do they cover? Who creates data and who funds their creation? Are data and their documentation accessible on Internet? In which format? Is access free? Given sheer number of potentially relevant datasets as well as their constant growth, finding orientation within complex, evolving landscapes of available cross-national data represents a major challenge in many fields of research. To give just two examples: In an extensive thematic data review, Todd Landman and Julia Hausermann (2003, i) identified more than 170 seminal efforts to measure democracy, human rights, and good governance. Similarly, in an overview over cross-national data on armed conflicts, Kristine Eck (2005, 3) inventoried almost 60 datasets, even while limiting herself to only the most prominent ones. Swimming in apparent data wealth, we run risk of drowning in numbers. In response to daunting costs of consumer information in disorderly markets of crossnational political data, some public institutions have begun to develop or commission data inventories that present systematic information about a broad range of cross-national data. Two fine examples are MacroDataGuide offered by Norwegian Social Science Data Services (NSD), which provides structured information about dozens of cross-national political datasets (www. …