Accuracy and inter-rater agreement among practitioners using quick thoracic ultrasonography to diagnose calf pneumonia.

Abstract

Thoracic ultrasonography (TUS) is a commonly used tool for on-farm detection of pneumonia in calves. Different scanning methods have been described, but the performance of novice practitioners after training has not been documented. In this study, 38 practitioners performed quick TUS (qTUS) on 18-23 calves each. Pneumonia was defined as lung consolidation 1cm or more in depth. Diagnostic parameters (accuracy [Acc], sensitivity [Se] and specificity [Sp]) were compared to those of an experienced operator. Cohen's kappa and Krippendorff's alpha (Kalpha) were determined. The potential effects of training and exam sessions on performance were evaluated. The average relative Se and Sp were 0.66 (standard deviation [SD] = 0.26; minimum [Min.]-Maximum [Max.] = 0-1) and 0.71 (SD = 0.19; Min.-Max. = 0.25-1), respectively. The average relative Acc was 0.73 (SD = 0.11; Min.-Max. = 0.52-0.96). Over all sessions, Cohen's kappa averaged 0.40 (SD = 0.24; Min.-Max. = 0.014-0.90) and Kalpha was 0.24 (95% confidence interval [CI]: 0.20-0.27), indicating 'fair' agreement. Calf age and housing influenced Se and Sp. Supervised practical training improved Se by 17.5% (95% CI: 0.01-0.34). The separate effects of calf age and housing could not be determined. This study showed that qTUS, like any other clinical skill, has a learning curve, and variability in performance can be substantial. Adequate training and certification of one's skill are recommended to assure good diagnostic accuracy.