Development and Testing for an Operative Competency Assessment Tool for Nasal Septoplasty Surgery

Abstract

Assessing surgical competency in otolaryngology is challenging, and residency programs are now responsible for ensuring the surgical competency of their graduates. Therefore, more objective assessment tools are being incorporated into the evaluation process. Objective structured assessment of technical skills (OSATSs) tools have been developed for multiple otolaryngology procedures. These include tonsillectomy, endoscopic sinus surgery, thyroidectomy, mastoidectomy, direct laryngoscopy, and rigid bronchoscopy. The purpose of this study was to develop and test a new assessment tool for septoplasty surgery and ensuring its feasibility, reliability, and construct validity. This study was designed to develop and test a valid, reliable, and feasible evaluation tool designed to measure the development of trainees' surgical skills in the operating room for septoplasty surgery. A new OSATSs-based instrument form for septoplasty was developed. During the study period of 2 years, 21 otolaryngology-head and neck surgery residents (ranging from postgraduate year 2 to 5) were evaluated intraoperatively by one faculty member obtaining a total of 175 evaluations. Surgical performance was rated using a seven-item task-specific checklist (TSC) and a global rating scale (GRS). The TSC assessed specific septoplasty technical skills, and the GRS assessed the overall surgical performance. Our tool showed construct validity for both components of the assessment instrument, with increasing mean scores with advancing clinical levels. Cronbach's α, a measure of internal consistency, was 0.911 for TSC and 0.898 for GRS. Strong correlation between the TSC and GRS was established (r = 0.955; p < 0.01). This study proved our educational tool to be a valid, reliable, and feasible method for assessing competency in septoplasty surgery. It can be integrated into surgical training programs to facilitate direct formative feedback. Assessing trainees' learning curves enables insight into their progression, ensuring their appropriate development.