Choice or Circumstance? Adjusting Measures of Foreign Policy Similarity for Chance Agreement

Abstract

The similarity of states' foreign policy positions is a standard variable in the dyadic analysis of international relations. Recent studies routinely rely on Signorino and Ritter's (1999, Tau-b or not tau-b: Measuring the similarity of foreign policy positions.International Studies Quarterly43:115–44)Sto assess the similarity of foreign policy ties. However,Sneglects two fundamental characteristics of the international state system: foreign policy ties are relatively rare and individual states differ in their innate propensity to form such ties. I propose two chance-corrected agreement indices, Scott's (1955, Reliability of content analysis: The case of nominal scale coding.The Public Opinion Quarterly19:321–5) π and Cohen's (1960, A coefficient of agreement for nominal scales.Educational and Psychological Measurement20:37–46) κ, as viable alternatives. Both indices adjust the dyadic similarity score for a large number of common absent ties. Cohen's κ also takes into account differences in individual dyad members' total number of ties. The resulting similarity scores have stronger face validity thanS. A comparison of their empirical distributions and a replication of Gartzke's (2007, The capitalist peace.American Journal of Political Science51:166–91) study of the ‘Capitalist Peace’ indicate that the different types of measures are not substitutable.