Interobserver and intraobserver reliability for 2 grading systems for gastric ulcer syndrome in horses.

Abstract

BackgroundGrading of equine gastric ulcer syndrome (EGUS) is undertaken in clinical and research settings, but the reliability of EGUS grading systems is poorly understood.Hypothesis/ObjectivesInvestigate interobserver and intraobserver reliability of an established ordinal grading system and a novel visual analog scale (VAS), and assess the influence of observer experience.AnimalsSixty deidentified gastroscopy videos.MethodsSix observers (3 specialists and 3 residents) graded videos using the EGUS Council (EGUC) system and VAS. Observers graded the videos three 3 for each system, using a cross‐over design with at least 1 week between each phase. The order of videos was randomized for each phase.MethodsInterobserver and intraobserver reliability were estimated using Gwet's agreement coefficient with ordinal weights applied (AC2) for the EGUC system and the intraclass correlation coefficient (ICC) for the VAS.ResultsUsing the EGUC system, interobserver reliability was substantial for squamous (AC2 = 0.69; 95% confidence interval [CI], 0.57‐0.80) and glandular mucosa (AC2 = 0.72; 95% CI, 0.70‐0.75), and intraobserver reliability was substantial for squamous (AC2 = 0.80; 95% CI, 0.71‐0.90) and glandular mucosa (AC2 = 0.80; 95% CI, 0.74‐0.86). Interobserver reliability using the VAS was moderate for squamous (ICC = 0.64; 95% CI, 0.31‐0.96) and poor for glandular mucosa (ICC = 0.35; 95% CI, 0.06‐0.64), and intraobserver reliability was moderate for squamous (ICC = 0.74; 95% CI, 0.62‐0.86) and glandular mucosa (ICC = 0.56; 95% CI, 0.39‐0.72).Conclusions and Clinical ImportanceThe EGUC system had acceptable intraobserver and interobserver reliability and performed well regardless of observer experience. Familiarity and observer experience improved reliability of the VAS.