Simulation Performance Scores Research Articles

Mastering multicasualty trauma care with the Trauma Non-technical Skills Scale.

Multicasualty events present complex medical challenges. This is the first study to investigate the role of nontechnical skills in prehospital multicasualty trauma care. We assessed the feasibility of using the Trauma Nontechnical Skills Scale (T-NOTECHS) instrument, which has not yet been investigated to evaluate these scenarios. We conducted an observational study involving military medical teams with Israel Defense Forces Military Trauma Life Support training to assess the T-NOTECHS' utility in predicting prehospital medical team performance during multicasualty event simulations. These teams were selected from a pool of qualified military Advanced Life Support providers. Simulations were conducted in a dedicated facility resembling a field setting, with video recordings to ensure data accuracy. Teams faced a single multicasualty scenario, assessed by two instructors, and were evaluated using a 37-item checklist. The T-NOTECHS scores were analyzed using regression models to predict simulation performance. We included 27 teams for analysis, led by 28% physicians and 72% paramedics. Interrater reliability for simulation performance and T-NOTECHS scores showed good agreement. Overall T-NOTECHS scores were positively correlated with simulation performance scores ( R = 0.546, p < 0.001). Each T-NOTECHS domain correlated with simulation performance. The Communication and Interaction domain explained a unique part of the variance ( β = 0.406, p = 0.047). Assessment and Decision Making had the highest correlation ( R = 0.535, p < 0.001). These domains significantly predicted specific items on the simulation performance checklist. Cooperation and Resource Management showed the least correlation with checklist items. This study confirms the T-NOTECHS' reliability in predicting prehospital trauma team performance during multicasualty scenarios. Key nontechnical skills, especially Communication and Interaction, and Assessment and Decision Making, play vital roles. These findings underscore the importance of training in these skills to enhance trauma care in such scenarios, offering valuable insights for medical team preparation. Diagnostic Tests or Criteria; Level III.

Virtual Reality–Enabled Resident Education of Lateral-Access Spine Surgery

OBJECTIVELateral access spine surgery has many benefits, but adoption has been limited by a steep learning curve. Virtual reality (VR) is gaining popularity and lends itself as a useful tool in enhancing neurosurgical resident education. We thus sought to assess if VR-based simulation could enhance the training of neurosurgery residents in lateral spine surgery. METHODSNeurosurgery residents completed a VR-based lateral spine module on lateral patient positioning and performing lateral lumbar interbody fusion (LLIF) using the PrecisionOS VR system on the Meta Quest 2 headset. Simulation occurred 1x/week every other week for a total of 3 simulations over 6 weeks. Pre- and post-intervention surveys as well as intra-simulation performance metrics were assessed over time. RESULTSThe majority of resident participants showed improvement in performance scores, including an automated PrecisionOS precision score, number of X-rays used within the simulation, and time to completion. All participants showed improvement in comfort with anatomic landmarks for lateral access surgery, confidence performing lateral surgery without direct supervision, and assessing fluoroscopy in spine surgery for hardware placement and image interpretation. Participant perception on the utility of VR as an educational tool also improved. CONCLUSIONVR-based simulation enhanced neurosurgical residents’ ability to understand lateral access surgery. Immersive surgical simulation resulted in improved resident confidence with surgical technique and workflow, perceived improvement in anatomical knowledge, and simulation performance scores. Trainee perceptions on virtual simulation and training as a curriculum supplement also improved following completion of VR training.

Effects of the Virtual Simulation Prebriefing-Debriefing Educational Strategy on Nursing Students’ Self-Efficacy and Virtual Simulation Performance: A Quasi-Experimental Study

BackgroundThrough the use of virtual simulations nursing students are exposed to a variety of clinical scenarios that may potentially improve their learning of competencies, increase their self-efficacy (confidence), and enhance their future clinical performance. MethodsGuided by the National League for Nursing Jeffries Simulation Theory, Bandura's self-efficacy theory, and the Healthcare Simulation Standards of Best PracticeTM, this quasi-experimental, two-group, pretest/post-test educational intervention study examined the effects of the synchronous Virtual Simulation Prebriefing-Debriefing strategy on Associate Degree Nursing students’ self-efficacy perceptions and virtual simulation performance scores in the care of patients experiencing a myocardial infarction. ResultsStudy results revealed statistically significant findings concerning the effectiveness of the Virtual Simulation Prebriefing-Debriefing strategy on students’ self-efficacy perceptions (in the cognitive, psychomotor, and affective domains) and virtual simulation performance scores. Results also support that there is a small positive correlation between students' self-efficacy perceptions and virtual simulation performance scores. This study also provided evidence that the study's educational strategy caused positive changes in students’ self-efficacy perceptions and virtual simulation performance scores regardless of students’ age, previous experience with video or computer gaming, or previous healthcare work experience. ConclusionFindings suggest that a high-quality virtual simulation experience that integrates the Healthcare Simulation Standards of Best PracticeTM, including structured prebriefing and debriefing strategies, can positively influence students’ self-efficacy perceptions and virtual simulation performance scores.

Improving Emergency Preparedness in Pediatric Primary Care Offices: A Simulation-Based Interventional Study.

Pediatric emergencies pose a challenge to primary care practices due to irregular frequency and complexity. Simulation-based assessment can improve skills and comfort in emergencies. Our aim was improving pediatric office emergency preparedness, as measured by adherence to the existing American Academy of Pediatrics policy statement, and quality of emergency care in a simulated setting, as measured by performance checklists. This was a single center study nested in a multicenter, prospective study measuring emergency preparedness and quality of care in 16 pediatric primary care practices and consisted of 3 phases: baseline assessment, intervention, and follow-up assessment. Baseline emergency preparedness was measured by checklist based on AAP guidelines, and quality of care was assessed using in-situ simulation. A report-out was provided along with resources addressing potential areas for improvement after baseline assessment. A repeat preparedness and simulation assessment was performed after a 6 to 10 month intervention period to measure improvement from baseline. Sixteen offices were recruited with 13 completing baseline and follow-up preparedness assessment. Eight of these sites also completed baseline and follow-up simulation assessment. Median baseline preparedness score was 70% and follow-up was 75.9%. Median baseline simulation performance scores were 37.4% and 35.5% for respiratory distress and seizure scenarios, respectively. Follow-up simulation assessment scores were 73% and 76.9% respectively (P=.001). Our collaborative was able to successfully improve the quality of care in a simulated setting in a group of pediatric primary care offices over 6 to 10 months. Future work will focus on expansion and improving emergency preparedness.

Learning optimal opioid prescribing and monitoring: a simulation study of medical residents.

ObjectiveHospitalized patients often receive opioids. There is a lack of consensus regarding evidence-based guidelines or training programs for effective management of pain in the hospital. We investigated the viability of using an Internet-based opioid dosing simulator to teach residents appropriate use of opioids to treat and manage acute pain.Materials and methodsWe used a prospective, longitudinal design to evaluate the effects of simulator training. In face-to-face didactic sessions, we taught 120 (108 internal medicine and 12 family medicine) residents principles of pain management and how to use the simulator. Each trainee completed 10 training and, subsequently, 5 testing trials on the simulator. For each trial, we collected medications, doses, routes and times of administration, pain scores, and a summary score. We used mixed-effects regression models to assess the impact of simulation training on simulation performance scores, variability in pain score trajectories, appropriate use of short- and long-acting opioids, and use of naloxone.ResultsTrainees completed 1582 simulation trials (M = 13.2, SD = 6.8), with sustained improvements in their simulated pain management practices. Over time, trainees improved their overall simulated pain management scores (b = 0.05, P < .01), generated lower pain score trajectories with less variability (b = −0.02, P < .01), switched more rapidly from short-acting to long-acting agents (b = −0.50, P < .01), and used naloxone less often (b = −0.10, P < .01).Discussion and conclusionsTrainees translated their understanding of didactically presented principles of pain management to their performance on simulated patient cases. Simulation-based training presents an opportunity for improving opioid-based inpatient acute pain management.

Simulation: Improving Knowledge and Retention of Infant Safe Sleep Practices

BackgroundPostdischarge, newborn caregivers are more likely to model the nursing behaviors exhibited in the hospital. Despite national recommendations, nurses' nonadherence to infant safe sleep best practices persists. MethodsA pre–post design was used to examine the effect of simulation-based learning experiences on the acquisition and retention of knowledge, behavior, and skills of nursing students regarding safe sleep practices. The sample consisted of 51 baccalaureate nursing students. ResultsStatistically significant differences between mean preintervention/postintervention written test scores, overall simulation performance scores, and safe sleep–specific simulation scores were found. ConclusionsThis study may provide the foundation for an evidenced-based approach to inform safe sleep practices to the future nursing workforce, which may influence caregiver practices at home.

Validation of a Novel Cognitive Simulator for Orbital Floor Reconstruction

The increasing focus on patient safety in current medical practice has promoted the development of surgical simulation technology in the form of virtual reality (VR) training designed largely to improve technical skills and less so for nontechnical aspects of surgery such as decision making and material knowledge. The present study investigated the validity of a novel cognitive VR simulator called Touch Surgery for a core maxillofacial surgical procedure: orbital floor reconstruction (OFR). A cross-sectional study was carried out on 2 groups of participants with different experience levels. Novice graduate dental students and expert surgeons were recruited from a local dental school and academic residency programs, respectively. All participants completed the OFR module on Touch Surgery. The primary outcome variable was simulator performance score. Post-module questionnaires rating specific aspects of the simulation experience were completed by the 2 groups and served as the secondary outcome variables. The age and gender of participants were considered additional predictor variables. From these data, conclusions were made regarding 3 types of validity (face, content, and construct) for the Touch Surgery simulator. Dependent-samples t tests were used to explore the consistency in simulation performance scores across phases 1 and 2 by experience level. Two multivariate ordinary least-squares regression models were fit to estimate the relation between experience and phase 1 and 2 scores. Thirty-nine novices and 10 experts who were naïve to Touch Surgery were recruited for the study. Experts outperformed novices on phases 1 and 2 of the OFR module (P < .001), which provided the measurement of construct validation. Responses to the questionnaire items used to assess face validity were favorable from the 2 groups. Positive questionnaire responses also were recorded from experts alone on items assessing the content validity for the module. Participant age and gender were not relevant predictors of performance scores. Construct, content, and face validities were observed for the OFR module on a novel cognitive simulator, Touch Surgery. Therefore, OFR simulation on the smart device platform could serve as a useful cognitive training and assessment tool in maxillofacial surgery residency programs.

Trauma Tactics: Rethinking Trauma Education for Professional Nurses.

According to the National Trauma Institute (2015), trauma accounts for more than 180,000 deaths each year in the United States. Nurses play a significant role in the care of trauma patients and therefore need appropriate education and training (L. ). Although several courses exist for trauma education, many nurses have not received adequate education in trauma management (B. ; L. ). Trauma Tactics, a 2-day course that focuses on high-fidelity human patient simulation, was created to meet this educational need. This descriptive study was conducted retrospectively to assess the effectiveness of the Trauma Tactics course. Pre- and postsurveys, tests, and simulation performance were used to evaluate professional nurses who participated in Trauma Tactics over a 10-month period. Fifty-five nurses were included in the study. Pre- and postsurveys revealed an increase in overall confidence, test scores increased by an average of 2.5 points, and simulation performance scores increased by an average of 16 points. Trauma Tactics is a high-quality course that provides a valuable and impactful educational experience for nurses. Further research is needed to evaluate the long-term effects of Trauma Tactics and its impacts on quality of care and patient outcomes.

Simulated environments with animated agents: effects on visual attention, emotion, performance, and perception

AbstractThis research assessed how emotive animated agents in a simulation‐based training affect the performance outcomes and perceptions of the individuals interacting in real time with the training application. A total of 56 participants consented to complete the study. The material for this investigation included a nursing simulation in which participants interacted with three animated agents. The results of this investigation indicated that both experienced and novice participants focused more visual attention time on the body of the animated agent than the other defined areas of interest in the simulated environment. The results also indicated that novice participants conveyed more neutral facial expressions during the interaction with the animated agents than experience participants. The results of the simulation performance scores indicated that novice participants achieved higher simulation performance scores on the simulation task than experienced participants. Lastly, the results of the agent persona instrument showed that experienced and novice participants perceived the animated agents as facilitators of learning, credible, human‐like and engaging.

Virtual reality job interview training in adults with autism spectrum disorder.

The feasibility and efficacy of virtual reality job interview training (VR-JIT) was assessed in a single-blinded randomized controlled trial. Adults with autism spectrum disorder were randomized to VR-JIT (n = 16) or treatment-as-usual (TAU) (n = 10) groups. VR-JIT consisted of simulated job interviews with a virtual character and didactic training. Participants attended 90 % of laboratory-based training sessions, found VR-JIT easy to use and enjoyable, and they felt prepared for future interviews. VR-JIT participants had greater improvement during live standardized job interview role-play performances than TAU participants (p = 0.046). A similar pattern was observed for self-reported self-confidence at a trend level (p = 0.060). VR-JIT simulation performance scores increased over time (R(2) = 0.83). Results indicate preliminary support for the feasibility and efficacy of VR-JIT, which can be administered using computer software or via the internet.

Board 314 - Research Abstract Simulation as a Tool for Improving Acquisition of Neonatal Resuscitation Skills (Submission #946)

Introduction/Background The first few minutes after birth are most critical for a newborn. Although most infants transition with ease, some require delivery room resuscitation.1 The quality of initial resuscitation may impact the infant’s short- and long-term outcome.2 Traditionally, neonatal providers are present at deliveries as soon as the need is recognized and delivery of the infant is imminent. However, in the event that the neonatal resuscitation team is not present, the obstetric team must initiate resuscitation. While obstetric nurses complete resuscitation training, obstetric physicians are often not required to complete neonatal resuscitation training. In order to fulfill this educational need we hypothesize that a short instructional session that teaches initial neonatal resuscitation steps with a simulation based curriculum will increase obstetric physician confidence, knowledge, performance and retention of neonatal resuscitation skills. Methods This was a prospective randomized, controlled trial of 33 obstetric residents. Subjects received baseline confidence, knowledge and clinical skills assessments. Subjects were then randomized to traditional lecture (n=14) or simulation-based (n=19) neonatal resuscitation education with a focus on initial steps. All instructional sessions were less than one hour in duration and both lecture and simulation interventions contained the same content. The simulation session required that the subject learn how to resuscitate a depressed neonate “hands on” with expert coaching. Follow-up assessments of confidence, knowledge and clinical skills occurred at intervals of three and six months post initial instruction. Confidence was reported based on a Likert scale of 1 (least confident) to 5 (most confident). Knowledge was assessed using seven multiple choice questions at the three intervals. Clinical assessments were performed in a simulated environment and were graded as yes/no if the subject completed individual tasks successfully in the following areas: preparation, initial resuscitation, ventilation, cardiovascular resuscitation. Total knowledge and clinical assessment scores were calculated. Statistical analysis was performed with paired T-tests to test individuals improvement and Student T-test for comparisons between groups. Results For both the lecture and simulation-based groups, confidence increased significantly at three and six months post instruction when compared to baseline (p<0.01). This increase in confidence was sustained as there were no significant differences between confidence at three and six months in both groups. Performance on knowledge assessment questions also improved significantly from baseline at three and six months post instruction for both groups when compared to baseline (p<0.01). Knowledge was maintained as there were no significant differences between knowledge scores at three and six months in both groups. Clinical assessments for the lecture group revealed no significant difference in successfully completed tasks between baseline and three months. However, significant improvement was demonstrated when six month performance was compared to baseline and 3 months (P<0.01). For the simulation group, clinical assessment performance was significantly improved at three and six months versus baseline (p<0.01) and this improvement was sustained with no significant difference between three and six month scores. Although there was an overall improvement in both groups, analysis with Student T-tests demonstrated that simulation performance scores were significantly higher than the lecture group at both three months (19.8 vs. 13.4) (p<0.01) and 6 months (19.1 vs. 17.2) (p<0.05). Conclusion A short, focused curriculum that teaches initial neonatal resuscitation steps via lecture or simulation enhances subject confidence, knowledge and skills. However, education that incorporates simulation is superior to lecture, resulting in more rapid acquisition of clinical skills and sustained improvements in overall clinical performance. Fortunately, an unanticipated depressed neonate is a relatively infrequent event; however, improved ability for obstetric providers to initiate a prompt and effective neonatal resuscitation after a brief simulation intervention, has the potential to greatly impact outcomes for these vulnerable newborns.

Integrative Learning Through Simulation and Problem-Based Learning

BackgroundAn integrative framework was used to guide the teaching, learning, and assessment approach taken in a final-year undergraduate nursing module. MethodStudents participated in three cycles of problem-based learning and associated simulations. An assessment rubric focused on patient assessment, clinical decision making, and technical and communication skills was used by students and assessors to score simulation performances. This study evaluated the trajectory of assessor and student self-assessed simulation performance scores over the course of the module. ResultsDescriptive statistics for assessors' scores show a mean improvement in all 4 competencies and for overall performance. Students' self-assessed scores do not show a mean improvement but are more closely aligned with assessor scores for later simulations.

Integrative Medicine Fellowships and Residencies Transforming the Healthcare Landscape

An integrative framework was used to guide the teaching, learning, and assessment approach taken in a final-year undergraduate nursing module.Students participated in three cycles of problem-based learning and associated simulations. An assessment rubric focused on patient assessment, clinical decision making, and technical and communication skills was used by students and assessors to score simulation performances. This study evaluated the trajectory of assessor and student self-assessed simulation performance scores over the course of the module.Descriptive statistics for assessors' scores show a mean improvement in all 4 competencies and for overall performance. Students' self-assessed scores do not show a mean improvement but are more closely aligned with assessor scores for later simulations.

Intelligence assessment with computer simulations

It has been suggested that computer simulations may be used for intelligence assessment. This study investigates what relationships exist between intelligence and computer-simulated tasks that mimic real-world problem-solving behavior, and discusses design requirements that simulations have to meet in order to be suitable for intelligence assessment. One hundred one participants took a test of inductive reasoning (BIS-K) and used the simulation MultiFlux [Kröner, S. (2001). Intelligenzdiagnostik per Computersimulation [Intelligence Assessment via computer simulation]. Münster: Waxmann.] designed to reduce the uncontrolled influence of prior knowledge, provide an evaluation-free exploration phase, and incorporate measures that are based on a theoretical model of simulation performance. Reliabilities of MultiFlux simulation performance scores were above .90, and the correlation of MultiFlux scores with BIS-K intelligence was, with r = .65 (adjusted r = .75), comparable to typical correlations among traditional intelligence tests. SEM analyses favored our theoretical performance model with three latent MultiFlux variables over a model with a single factor.

The Tournament Concept in Assessment

This article reports on two studies that were conducted to determine whether (a) total enterprise simulations are suitable for assessing business skills and (b) the tournament concept can be effectively applied to such simulations. The first study involved 141 senior business students; the second, 74. The results are consistently favorable on both questions. The studies also show that the free-rider problem that may have previously undermined the validity of simulation performance scores can be overcome by incorporating a system of individual performance scores, and that a progressively difficult simulation experience can keep participants challenged for 30 decision periods.

Exploring Teaching Techniques via a Microcomputer Simulation

ABSTRACT This paper describes a study designed to define the principal techniques that pre‐service teachers apply as they address the challenges encountered in completing the current Curry Teaching Simulations. Factor analysis of simulation performance scores yielded three clearly identifiable performance factors. The three factors addressed the techniques which defined teachers’ misbehavior intervention, instructional delivery, and activity sequencing, respectively. Regression analyses were employed to assess the relationship of each of the three factors to personality and demographic variables.