The importance of technical skills assessment during an airway foreign body removal course

Abstract

PurposeSurgical simulation has proven useful in training for high-acuity, low frequency events such as airway foreign body removal (AWFBR). Studies have supported the role of simulation to improve trainee confidence and technical proficiency, but rigorous methodology is often lacking.Through a pilot study, we sought to evaluate the feasibility and utility of two-view video capture for rigorous assessment of trainee skill in an educational course setting. MethodsParticipants were asked to perform 1) self-assessment surveys using 5- point Likert scale questions and 2) rigid bronchoscopy with AWFBR on intubation trainers, at the beginning and end of a multi-institution airway course. Video was collected from endoscopic and wide-angle body cameras. The videos were blinded and evaluated in random order by two expert reviewers using a 5-point pediatric airway endoscopy specific objective structured assessment of technical skills (OSATS) instrument. ResultsFourteen trainees submitted pre- and post-course surveys, and eight of these also had complete video data. Faculty feedback indicated the importance of pre- and post-course AWFBR recordings for real-time trainee feedback and post-course curricular refinement. Survey data showed an increase in confidence for AWFBR from 2.0 to 3.3 (p = 0.05). Average OSATS scores increased from 1.84 to 2.58 but this did not reach significance (p = 0.51). Paired improvements in confidence exceeded improvements in OSATS scores (1.29 vs. 0.18, p = 0.058). ConclusionsRigorously evaluated two-view video capture was feasible in an educational course setting. The course resulted in improvements in confidence to a greater degree than OSATS scores. This supports the importance of assessing course impact and refining curricula using all available data including objectively assessing technical skills.