Validity and reliability of a sensor-enabled intubation trainer: a focus on patient-centered data

Abstract

BackgroundPrior work using simulation for assessing intubation skills has largely focused on the use of observer-generated performance measures in the form of checklists and global ratings scales. PurposeThe purpose of our work was to investigate whether patient-centered simulation data could be used to quantify learner's performance during direct laryngoscopy. MethodsWe designed a pretest/posttest prospective intervention study of residents' (n = 25) intubation skills. ResultsWhen assessing validity, all of the patient-centered simulation variables showed significant correlations with the previously validated observer-generated performance measures (r = 0.331–0.463, P ≤ 0.001). When assessing reliability, there were significant correlations between all of the sensor variables, confirming moderate to high inter-item reliability (r = 0.259–0.794, P ≤ 0.05). The observer-generated performance measures showed significant improvement in use of the Macintosh blade (T1 = 2.10/5.00, T2 = 3.64/5.00, P = 0.001). However, this was not the case for the Miller blade (T1 = 1.30/5.00, T2 = 1.75/5.00, P = 0.119). Overall, the patient-centered simulation variables provided a high level of detail regarding performance improvement areas. ConclusionThis study presents a multilevel analysis of sensor-generated simulation data. As the sensors provide sound, formative data regarding patient contact, the outputs may be used for specific criterion measures and detailed performance feedback.