Abstract
R package krippendorffsalpha provides tools for measuring agreement using Krippendorff's Alpha coefficient, a well-known nonparametric measure of agreement (also called inter-rater reliability and various other names). This article first develops Krippendorff's Alpha in a natural way, and situates Alpha among statistical procedures. Then the usage of package krippendorffsalpha is illustrated via analyses of two datasets, the latter of which was collected during an imaging study of hip cartilage. The package permits users to apply the Alpha methodology using built-in distance functions for the nominal, ordinal, interval, or ratio levels of measurement. User-defined distance functions are also supported. The fitting function can accommodate any number of units, any number of coders, and missingness. Bootstrap inference is supported, and the bootstrap computation can be carried out in parallel.
Highlights
Krippendorff’s α (Hayes and Krippendorff, 2007) is a well-known nonparametric measure of agreement (i.e., consistency of scoring among two or more raters for the same units of analysis (Gwet, 2014))
In Section 2.2.2, we show that α is a type of multiresponse permutation procedure
We described Krippendorff’s α methodology for measuring agreement and illustrated the use of R package krippendorffsalpha
Summary
Krippendorff’s α (Hayes and Krippendorff, 2007) is a well-known nonparametric measure of agreement (i.e., consistency of scoring among two or more raters for the same units of analysis (Gwet, 2014)). The remainder of this article is organized as follows. In Section 2.2.1, we first develop a special case of Krippendorff’s α (call it αSED) in a well-known parametric setting, and we present α in its most general (i.e., nonparametric) form. In Section 2.2.2, we show that α is a type of multiresponse permutation procedure. In Section 2.2.3, we generalize αSED in a fully parametric fashion, arriving at Sklar’s ω.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
