59 Inter-rater reliability of equine nutrition screening tools in a clinical setting

Abstract

Malnutrition, a nutritional imbalance capable of occurring within any body condition, continues to be underrecognized and undertreated in presumed healthy and hospitalized equids. In veterinary medicine, no agreed “gold-standard” exists for a malnutrition diagnosis in equine patients, exaggerated by a lack of screening tools (ST) being performed on each patient at every veterinary visit. Our objective was to assess inter-rater reliability (IRR) among trained inter-disciplinary professionals when performing suggested routine nutrition screening tools in equine patients. Between March and November 2022, 90 horses admitted to the University of Tennessee's Large Animal Hospital were reviewed by up to 5 trained individuals: 3 board-certified veterinarians (2 equine internal medicine specialists, 1 theriogenologist), 1 veterinary resident, 1 PhD equine nutritionist, and 1 nutrition specialty veterinary nurse. Raters underwent one training session together for ST before study enrollment including; body condition score (BCS), muscle atrophy scoring system (MASS), cresty neck score (CNS), and equine comfort assessment scale (ECAS). Enrolled horses encompassed the following ST ranges; BCS 1–9, MASS 1–4, CNS 0–4, and ECAS 0–3. Raters performed ST evaluations individually and were blinded to other rater's scores. Statistical analysis for IRR were performed using Gwet's AC1 (Stata17 SE, College Station, TX), results reported as coefficient (percent agreement) correspondingto benchmarks from substantial (0.6000–0.8000) to almost perfect (0.8000–1.0000). Among all raters (up to 5), almost perfect reliability occurred within BCS 0.94 (98.6%) with BCS range of 1–9, ECAS 0.89 (95.1%) with a range of 0–3, MASS Back 0.76 (91.3%), and substantial reliability for MASS Hind 0.80 (92.0%), CNS 0.77 (93.2%), and MASS Neck 0.76 (91.3%). Due to rater availability, ultimately, 2 raters (1 PhD equine nutritionist, and 1 nutrition specialty veterinary nurse) screened majority of the horses (n = 52). Similar IRR was detected among the 2 raters with almost perfect reliability within BCS 0.95 (98.8%), ECAS 0.90 (95.3%), MASS Hind 0.88 (94.4%), and MASS Neck 0.81 (92.7%) with substantial reliability among MASS Back 0.78 (89.9%), and CNS 0.76 (92.7%). Our data suggests that these 5 trained, inter-professional raters reliably agreed on all ST when evaluating horses. While the MASS Back, MASS Neck and CNS had the lowest agreement and most variability among raters, these were still considered substantially reliable. This suggests that these ST, when using the same trained raters, could be a reliable part of a larger development in performing equine nutrition screenings.