Assessing the interrater and intrarater reliability of subglottic stenosis grading systems

Abstract

BackgroundSubglottic stenosis (SGS) is a condition leading to narrowing of the upper airway which can lead to dyspnea and life-threatening airway obstruction. Although other proposed grading systems exist, the Cotton Myer (CM) and percent stenosis systems are the most widespread in clinical practice. Despite this, the CM system has not yet been validated for visual assessment of SGS. ObjectiveTo determine the interrater and intrarater reliability of the CM grading system among a cohort of physicians who manage patients with SGS. MethodsAn online survey created with videos of tracheoscopies from 20 adult patients with subglotticstenosis (SGS) was sent individually to 9 expert physicians from various medical specialties, all of whom managed patients with SGS. Physicians were asked to view the 20 tracheoscopy videos and assess both the percent stenosis and Cotton Myer (CM) grade of each patient. After a period of 4 weeks, the physicians were sent the same survey of the 20 tracheoscopy videos. The interrater and intrarater reliability was calculated using the intraclass correlation coefficient (ICC), a measurement used to evaluate the reliability (the extent to which a measurement can be replicated) of two or more raters measuring the same subject. ResultsOverall, CM and percent stenosis systems were found to have an ICC of 0.94 and 0.90 within the domain of interrater reliability, respectively, and ICC of 0.71 and 0.81 within the domain of intrarater reliability, respectively. ConclusionOur findings suggest that the CM and percent stenosis grading systems remain a valid clinical tool to measure and communicate the severity of airway obstruction in SGS.