Interrater reliability of endoscopic parameters following sinus surgery

Abstract

To determine the interrater reliability of a set of postoperative endoscopic scoring parameters in patients with chronic rhinosinusitis who have undergone endoscopic sinus surgery (ESS). Prospective cohort with retrospective review. One hundred twenty video-endoscopic evaluations in 20 subjects recorded at 14, 30, and 45 days after ESS were scored in real time by the clinical investigators who performed the endoscopies and recorded the videos and retrospectively by an independent panel of four sinus surgeons who were blinded to all information. The scoring parameters included categoric grading for adhesion formation and middle turbinate position and continuous grading (visual analog scale) for degree of inflammation and crusting. Interrater reliability of the panel members was assessed using the Fleiss kappa test, bias index and prevalence index for categoric data, and the Shrout-Fleiss test for continuous data. The level of agreement between the panel and the real-time clinical investigator was also assessed. For categoric variables, strong agreement between raters on the panel was found for both middle turbinate position (kappa=0.499, prevalence index=0.925) and adhesions (kappa=0.364, prevalence index=0.829). For continuous data, good agreement between raters was found for both inflammation (reliability coefficient=0.554) and crusting (reliability coefficient=0.620). Real-time investigator scoring and panel scoring showed strong agreement. These results suggest that the endoscopic scoring parameters assessed (middle turbinate position, adhesions, inflammation, and crusting) have acceptable interexaminer reproducibility and are suitable for evaluating ESS outcomes in the postsurgical period.