Board 330 - Research Abstract Simulation-Based Assessment of ECMO Clinical Specialists (Submission #442)

Abstract

Introduction/Background Extracorporeal Membrane Oxygenation [ECMO] is a life-saving measure with significant risks of morbidity and mortality for critically ill patients in cardiac or pulmonary failure. Clinical specialists in the management of ECMO patients provide continuous bedside monitoring and are the first line of defense in recognizing and responding to acute and potentially catastrophic events. However, since such events are infrequent and unpredictable, an ECMO specialist with many years of experience may never have encountered one. The purpose of this study was to create simulated scenarios of critical events to enhance the training of ECMO specialists. A second goal was to assess their performance in recognizing and responding to the simulated events. The hypothesis was that more experienced individuals would outperform the less experienced. The simulations provided a hands-on, real-time approach to the challenging and complex nature of bedside ECMO emergencies. Methods Eight scenarios, designed to reflect acute events an ECMO specialist would be expected to manage included hyperventilation, air entrainment, ventricular tachycardia, hypovolemia causing venous occlusion, agitation with arterial occlusion, pneumothorax, bleeding and hemodilution. The scenarios were created and piloted by perfusionists, critical care physicians and the ECMO nurse coordinator. These scenarios represent events that have occurred while a patient was on ECMO support. Twenty five ECMO clinical specialists consented to participate in this IRB-approved study. The subjects were oriented to the manikin and the study format and participated individually. They had five minutes to complete each scenario and an in-depth debriefing was held at the Conclusion of the session. The sessions were captured on an audiovisual system. Two trained raters scored for completion of key actions and assessed a global score. Results All twenty five participants completed the scenarios. Thirteen (52%) had less than one year of experience on the ECMO team. The generalizability coefficients were moderate for both the checklist (0.41) and global scores (0.43). Large proportions of variance were attributable to the Person x Task (P x T) interaction, suggesting that performance in one scenario was poorly predicted by performance in another. Relatively little variance was attributable to the rater, or associated interactions, indicating the raters were reasonably consistent in the assignment of scores. The standardized inter-rater reliability coefficient were comparable for the checklist scores (r=0.72) and the global scores (r=0.70). For the ANOVA of both the checklist and global scoring, the group (experience) by scenario interaction was not significant indicating that any performance differences between ECMO groups were reasonably consistent across scenarios. There was a main effect attributable to group by checklist (F=4.3, p<0.05) and global score (F=8.4, p<0.015) indicating that the more experienced ECMO group outperformed the less experienced group. Participants with >1 year experience (mean=77.5, SD=17.9) outperformed those with <=1 year experience (mean=73.1, SD=14.1) by checklist scoring as well as in their global score [>1 year: (mean=7.2, SD=1.5) versus <=1 year: (mean=6.6, SD=1.4)]. There was also a main effect attributable to scenario based on checklist scores (F=6.9, p<0.01), as well as the global rating (F=4.1, p<0.01) indicating that the scenarios were not of equal difficulty. Conclusion ECMO simulation scenarios were created for acute skills training and evaluation of ECMO clinical specialists. Regarding reliability, the study revealed a great deal of task specificity, which is not unexpected with a heterogeneous group of tasks to be accomplished. Construct validity was supported by the experienced specialists outperforming the less experienced. It was also supported by participant feedback following the sessions. In their evaluations, the participants felt that the simulation scenarios were realistic, represented events that could occur in practice and were appropriate for training. These preliminary data support the simulation of ECMO critical events for training and evaluation of clinical specialists.