Validation Study on Intraoperative Video Recording as an Objective Assessment Tool in ENT Surgery

Abstract

Objectives: (1) Determine the construct validity and the reliability of video assessment scoring of myringotomy and grommet insertion. (2) Measure the performance (by rating) of participants compared to operative time. Methods: Study design was a single blinded (raters) video assessment, conducted in a tertiary care university hospital August to October 2013. Participants were consultant and trainee (Specialty Registrar [StR] and Core Trainee [CT]) ear, nose, and throat (ENT) surgeons performing a total of 30 consecutive index procedures. Two raters at ENT Consultant level with a subspecialty interest in otology who did not take part in the study were invited to score results. Results: A strong correlation between scores by the 2 blinded raters was demonstrated (rho = 0.748; P < .001). Median scores (/45) for each group were: CT 25.5 (interquartile range [IQR] 21.13-31.25), SpR 33 (IQR 24.88-35), and consultant 40 (IQR 35.38-42.63). Kruskal-Wallis test analysis showed statistically significant mean rank scores between the 3 different levels of experience (H = 12.77, P = .002). Multiple group comparisons indicated a significant difference between CT and consultant groups ( P < .001) and StR and consultant groups ( P = .007). Analysis of the time taken between the experience groups demonstrated a difference (H = 8.689, P = .013), although individual intergroup comparisons indicated this was only significant between CT and consultant groups ( P = .004). There was a significant negative correlation (rho = –0.842; P < .001) between time taken for procedure and score achieved. Conclusions: Video assessment of this procedure may represent a valid, feasible tool for use in summative and formative assessments of trainee ENT surgeons. Remote scoring of assessment procedures minimizes bias and enables blinding of raters.