How (not) to measure democracy

Abstract

Measures of democracy are regularly employed in the statistical analysis of economic, political, and social policy. This paper reviews the measures" setup, strength, and weaknesses across the three most prominent democracy datasets: PolityIV, Freedom House, and Varieties of Democracy. The measures developed by the Varieties of Democracy project outperform Polity2 and Freedom House Index with respect to the underlying definition and measurement scale, as well as the theoretical justification of the aggregation procedure. The three indices display a high level of agreement for those observations included in all three datasets. The most substantial differences between the indices lie in the indices’ coverage, i.e. in their non-missing observations (in Polity2 coding, for example, years during which a country is occupied by foreign powers constitute missing values), the availability of disaggregate data and the above mentioned key areas. This paper clarifies when to proceed with caution, but for the most part advocates the use of Varieties of Democracy in the statistical analysis of democracy.