Interrater and intrarater agreement on heart murmurs

Abstract

Objective To investigate interrater and intrarater agreement between physicians and medical students on heart sound classification from audio recordings, and factors predicting agreement with a reference classification. Design Intra- and interrater agreement study. Subjects Seventeen GPs and eight cardiologists from Norway and the Netherlands, eight medical students from Norway. Main outcome measures Proportion of agreement and kappa coefficients for intrarater agreement and agreement with a reference classification. Results The proportion of intrarater agreement on the presence of any murmur was 83% on average, with a median kappa of 0.64 (range k = 0.09–0.86) for all raters, and 0.65, 0.69, and 0.61 for GPs, cardiologist, and medical students, respectively. The proportion of agreement with the reference on any murmur was 81% on average, with a median kappa of 0.67 (range 0.29–0.90) for all raters, and 0.65, 0.69, and 0.51 for GPs, cardiologists, and medical students, respectively. Distinct murmur, more than five years of clinical practice, and cardiology specialty were most strongly associated with the agreement, with ORs of 2.41 (95% CI 1.63–3.58), 2.19 (1.58–3.04), and 2.53 (1.46–4.41), respectively. Conclusion We observed fair but variable agreement with a reference on heart murmurs, and physician experience and specialty, as well as murmur intensity, were the factors most strongly associated with agreement. Key points: Heart auscultation is the main physical examination of the heart, but we lack knowledge of inter- and intrarater agreement on heart sounds. • Physicians identified heart murmurs from heart sound recordings fairly reliably compared with a reference classification, and with fair intrarater agreement. • Both intrarater agreement and agreement with the reference showed considerable variation between doctors • Murmur intensity, more than five years in clinical practice, and cardiology specialty were most strongly linked to agreement with the reference.