Can Self-Reported Confidence be used as an Indicator for Performance? A Multi-Center Study of Simulation Training for Acute Ischemic Stroke (P9-2.002)

Abstract

<h3>Objective:</h3> To assess changes in neurology residents’ confidence in managing acute ischemic stroke after a simulated case and explore if confidence scores correlate with knowledge and performance. <h3>Background:</h3> Prior studies have shown improvement in trainee confidence after simulation interventions. Few studies have assessed the effect of simulation training on neurology resident confidence in acute stroke management. <h3>Design/Methods:</h3> We conducted a multicenter, simulation-based study of neurology residents’ confidence managing a case of acute ischemic stroke complicated by alteplase-related hemorrhage and cerebral herniation. Participants rated their confidence in managing acute ischemic stroke on a 7-point Likert scale before and after the simulation. Neurology resident knowledge was assessed before the simulation on a multiple-choice test. Performance in the simulation was assessed using a checklist of 22 critical action items, supported by previously published validity evidence. The primary outcome was confidence before and after the simulation, assessed by paired samples t-test. The secondary outcome was correlation of confidence with knowledge assessed by multiple-choice test and performance assessed by simulation checklist score. <h3>Results:</h3> Thirty-six neurology residents (26 post-graduate year 2 [PGY-2], 7 PGY-3, 3 PGY-4) participated in the scenario. Residents performed a mean sum of 13.1 (standard deviation [SD] 2.3) out of 22 (60%) critical action items. Resident confidence increased after the simulation (post-simulation mean = 5.6 (1.1) vs. pre-simulation mean 4.0 (1.7)), <i>p</i> &lt; .001). Neither pre- nor post-simulation confidence correlated with scores on the multiple-choice test (r = 0.02, <i>p = .927</i> and r = −.09, <i>p</i> = .610, respectively). Similarly, neither pre-simulation confidence nor post-simulation confidence was correlated with performance in the simulation (r = 0.123, <i>p</i> = .502 and r = −0.015, <i>p</i> = .936, respectively). <h3>Conclusions:</h3> Simulation training increases neurology residents’ confidence in managing acute ischemic stroke. Confidence should not be used as a surrogate for knowledge and performance. <b>Disclosure:</b> Dr. Pergakis has nothing to disclose. Dr. Albin has received personal compensation in the range of $5,000-$9,999 for serving as an Editor, Associate Editor, or Editorial Advisory Board Member for Continuum: Life Long Learning in Neurology. Dr. Albin has received research support from American Academy of Neurology. Dr. Albin has received publishing royalties from a publication relating to health care. Dr. Albin has received personal compensation in the range of $500-$4,999 for serving as a Instructor with Resuscitation Leadership Academy. Mr. Neustein has nothing to disclose. The institution of Dr. Morris has received research support from American Board of Psychiatry and Neurology. The institution of Dr. Morris has received research support from UMB ICTR through NIH NCATS&amp;CTSA grant 1UL1TR003098. The institution of Dr. Morris has received research support from University of Maryland Medical Center.