Evaluation Tools in Simulation Learning: Performance and Self-Efficacy in Emergency Response

Abstract

Introduction High performance in emergency response is critical for high-quality patient outcomes but is not assessed in some traditional critical care nurse internships. Recognition of the opportunity to assess these skills in a simulated environment is growing, but validated measurement tools are needed. This study develops and evaluates two tools to assess confidence in emergency response and psychomotor performance during a simulated emergency response experience. Method Using constructs from basic life support and advanced cardiac life support, an emergency response performance tool (ERPT) and a confidence tool were developed. The ERPT consisted of 11 categorical items and six timed tasks. The confidence tool comprised 17 items and required participants to rate their confidence on a scale from 0% to 100%. A total of 41 registered nurses were assigned to one of three groups: (a) those experienced in critical care without formal training, (b) those with < 13 months experience with formal training, and (c) those with no critical care experience or training. All participants completed the knowledge test and confidence tool. From the 41 participants, 12 were selected randomly and scheduled to participate in one simulation event and then rate their confidence a second time. Trained raters viewed video recordings of the participants' performance and rated them using the ERPT. Results The statistical findings from this study support reliability and validity for the ERPT performance measure and reliability and internal consistency of the confidence tool. Interrater reliability on the ERPT was high for the majority of items. Comparisons among the three groups found significant differences in knowledge and confidence among the groups. Conclusions The confidence tool and the ERPT show a basic level of validity, reliability, and usability. Findings from the study have guided the development of more-robust performance and confidence instruments to use in a future comparison study to examine three different simulation teaching methodologies.