Should We Agree to Disagree? An Evaluation of the Inter-Rater Reliability of Gait Quality Traits in Franches-Montagnes Stallions

Abstract

Gait quality, that is, the way horses move according to functional and aesthetic principles, englobes many traits that are scored by experts during breeding competitions. The experts can score a trait on a subjective valuating (SV) scale or on a linear profiling (LP) scale representing the biological extremes of the population. However, the reliability of the appraisal of gait quality traits has not been extensively evaluated. In this study, seven breed experts appraised the walk and trot quality of 24 Franches-Montagnes stallions presented in hand on a sand track. Inter-rater reliabilities of six traits (five SV and one LP) at the walk and eight traits (five SV and three LP) at the trot were estimated with intraclass correlation coefficients (ICCs). The inter-rater reliabilities were poor (ICC < 0.50). The scale anchoring varied between experts, and the variance of scores was low. There were no systematic differences in inter-rater reliability between LP and SV traits. Future studies should determine whether the inter-rater reliabilities may be increased by a more precise definition of the scores within each trait to improve the absolute agreement between experts, by a more uniform scale anchoring between experts, and by decreasing the number of scale items. However, considering the inherent limitations of the human eye in observing high-speed movement, the use of a field-applicable kinematic measurement system may support breeding experts in the appraisal of gait quality traits in the future.