Kappa values for focused abdominal sonography for trauma examination interrater reliability based on anatomic view and focused abdominal sonography for trauma experience level

Abstract

Study objectives: Accuracy and precision are important components of focused abdominal sonography for trauma (FAST) examination interpretation. The accuracy of FAST examinations is known to depend on physician experience. However, the effect of physician experience on precision of FAST examination interpretation has not been previously reported. This study's objective is to analyze the precision (as judged by κ coefficient of interrater reliability) of emergency physician FAST interpretation as a function of emergency physician experience level with FAST interpretation for the 4 standard FAST views.Methods: A prospective, nonrandomized, comparative study of emergency physician FAST examination interpretation was conducted using video recordings of the 4 standard anatomic views from 5 patients' FAST examinations (20 total video clips). Each recording had some normal view(s) and some abnormal view(s). Free fluid presence was judged by a blinded surgical sonologist with 20 years of ultrasonographic experience and backed by computed tomographic scan or laparotomy results. κ Values for interpretations by 14 more-experienced emergency physicians (More Exp. ½; career FAST examinations: mean 394±303) were first compared against interpretations by 14 less-experienced emergency physicians (Less Exp. ½; career FAST examinations: mean 47±10), divided at the 50th percentile (Most versus Least Experienced half (MvL½)]. Next, κ values for each pair of emergency physicians' interpretations were assessed (P, pairs), pairing the most experienced emergency physician versus the next most experienced, the third versus fourth most experienced, and so on, for all emergency physicians (Pairs, All) and between the more and less experienced halves (Pairs, More Exp. ½; and Pairs, Less Exp. ½), to further assess the effect of emergency physician experience on κ. κ Agreement was judged excellent (κ>0.8), good (0.8≥κ>0.6), moderate (0.6≥κ>0.4), fair (0.4≥κ>0.2), or poor (κ≤0.2).Results: Overall FAST accuracy for More Exp. ½ was 92.5% (259/280) and for Less Exp. ½; was 83.2% (233/280). κ Values are shown in the Table.Conclusion: κ Values comparing MvL½ of emergency physicians showed good interrater agreement for all 4 anatomic views combined, right upper quadrant, and left upper quadrant views, and moderate agreement for pericardial and suprapubic views. Paired analysis nested for more-experienced versus less-experienced emergency physicians showed that most interobserver disagreement for all but the left upper quadrant view was present in the less-experienced emergency physician subgroup. The amount of previous emergency physician experience with interpretation of the FAST examination strongly influenced the precision of emergency physician interpretation of the FAST examination. Only more experienced pairs of emergency physicians uniformly achieved excellent or good κ values for 4 view FAST interpretations.Table, abstract 103.ComparisonMvL1/2Pairs, More Exp. 1/2Pairs, Less Exp. 1/2Pairs, AllAll 4 views combined0.630.740.510.63Pericardial0.440.640.390.51Right upper quadrant0.720.960.660.77Left upper quadrant0.630.660.660.66Suprapubic0.530.680.220.45