Reliability of body condition scoring of sheep for cross-farm assessments

Abstract

In order to use body condition scoring as a cross-farm assessment tool, it is important that different assessors can consistently grade sheep along the same scale. This paper examined the intra- and inter-assessor reliability of three trained and experienced assessors who independently assessed the body condition of 141 Lleyn, Cambridge-cross and Welsh Mule-crossbred ewes using full- or half-point scoring precision of a six-point ordinal scale. Assessor reliability was evaluated using percentage (%) agreement, weighted kappa (κw), and Kendall's coefficient of concordance (W). Kruskal–Wallis one-way analysis of variance and graphical representation of the data were also used to examine for assessor bias. The on-farm studies found that the intra-assessor reliability of an assessor, who provided training in the scoring method, was higher when half- (96%, κw 0.7, W 0.7) rather than full-unit scoring precision (79%, κw 0.6, W 0.6) was used. Similarly, a higher level of inter-observer agreement was found when two additional assessors applied half- (94%, κw 0.6, W 0.7) rather than full-unit scores (93%, κw 0.4, W 0.4). Consequently, the effect of a brief re-calibration exercise on the between-observer agreement for the assessment of full-unit body condition scores (BCS) was examined. Prior to the exercise, the paired agreement between two assessors and the trainer ranged from 68 to 78%, κw 0.3–0.5, and W 0.4–0.5. Following the re-calibration exercise, the level of inter-observer agreement increased to 75–93%, κw 0.4–0.7, and W 0.4–0.6. No significant effect of assessor bias was found (p>0.05). However, most sample sheep were identified within the mid-range of body condition (BCS 2–3), which affected the analysis and interpretation of reliability data. Overall, the results suggested that trained and experienced assessors reliably scored the body condition of sheep using both half- and full-unit scores, and that a period of re-calibration may offer a feasible means of maintaining the consistency of cross-farm assessments performed by different assessors.