Simulation-based Mastery Learning Curriculum Research Articles

Use of a simulation-based mastery learning curriculum for neurology residents to improve the identification and management of status epilepticus

BackgroundAppropriate and timely treatment of status epilepticus (SE) decreases morbidity and mortality. Therefore, skill-based training in the identification and management of SE is crucial for clinicians. ObjectiveThe objective of the study was to develop and evaluate the impact of a simulation-based mastery learning (SBML) curriculum to train neurology residents on the identification and management of SE. MethodsWe used pretest–posttest design with a retention test on SE skills for this study. Neurology residents in the second postgraduate year (PGY-2) were eligible to participate in the SE SBML curriculum. Learners completed a baseline-simulated SE skills assessment (pretest) using a 26-item dichotomous skills checklist. Next, they participated in a didactic session about the identification and management of SE, followed by deliberate skills practice. Subsequently, participants completed another skills assessment (posttest) using the same 26-item checklist. All participants were required to meet or exceed a minimum passing standard (MPS) determined by a panel of 14 SE experts using the Mastery Angoff standard setting method. After meeting the MPS at posttest, participants were reassessed during an unannounced in situ simulation session on the medical wards. We compared pretest with posttest simulated SE skills performance and posttest with reassessment in situ performance. ResultsThe MPS was set at 88% (23/26) checklist items correct. Sixteen neurology residents participated in the intervention. Participant performance improved from a median of 44.23% (Interquartile range (IQR): 34.62–55.77) at pretest to 94.23% (IQR: 92.13–100) at the posttest after SBML (p < .001). There was no significant difference in scores between the posttest and in situ test up to 8 months later (94.23%; IQR: 92.31–100 vs. 92.31%; IQR: 88.46–96.15; p = .13). ConclusionsOur SBML curriculum significantly improved residents' SE identification and management skills that were largely retained during an unannounced simulated encounter in the hospital setting.

Promoting Readiness for Residency: Embedding Simulation-Based Mastery Learning for Breaking Bad News Into the Medicine Subinternship.

It is challenging to add rigorous, competency-based communication skills training to existing clerkship structures. The authors embedded a simulation-based mastery learning (SBML) curriculum into a medicine subinternship to demonstrate feasibility and determine the impact on the foundational skill of breaking bad news (BBN). All fourth-year students enrolled in a medicine subinternship at Northwestern University Feinberg School of Medicine from September 2017 through August 2018 were expected to complete a BBN SBML curriculum. First, students completed a pretest with a standardized patient using a previously developed BBN assessment tool. Learners then participated in a 4-hour BBN skills workshop with didactic instruction, focused feedback, and deliberate practice with simulated patients. Students were required to meet or exceed a predetermined minimum passing standard (MPS) at posttest. The authors compared pretest and posttest scores to evaluate the effect of the intervention. Participant demographic characteristics and course evaluations were also collected. Eighty-five students were eligible for the study, and 79 (93%) completed all components. Although 55/79 (70%) reported having personally delivered serious news to actual patients, baseline performance was poor. Students' overall checklist performance significantly improved from a mean of 65.0% (SD = 16.2%) items correct to 94.2% (SD = 5.9%; P < .001) correct. There was also statistically significant improvement in scaled items assessing quality of communication, and all students achieved the MPS at mastery posttest. All students stated they would recommend the workshop to colleagues. It is feasible to embed SBML into a required clerkship. In the context of this study, rigorous SBML resulted in uniformly high levels of skill acquisition, documented competency, and was positively received by learners.

Patient, Caregiver, and Clinician Perceptions of Ventricular Assist Device Self-care Education Inform the Development of a Simulation-based Mastery Learning Curriculum.

Patients who undergo ventricular assist device (VAD) implantation and their caregivers must rapidly learn a significant amount of self-care skills and knowledge. The aim of this study was to explore patient, caregiver, VAD coordinator, and physician perspectives and perceptions of existing VAD self-care training to inform development of a simulation-based mastery learning (SBML) curriculum to teach patients and caregivers VAD self-care skills and knowledge. We conducted semistructured, in-person interviews with patients with a VAD, their caregivers, VAD coordinators, and physicians (cardiac surgeons, an infectious disease physician, and advanced heart failure cardiologists). We used a 2-cycle team-based iterative inductive approach to coding and analysis. We interviewed 16 patients, 12 caregivers, 7 VAD coordinators, and 11 physicians. Seven major themes were derived from the interviews including (1) identification of critical curricular content, (2) need for standardization and assessment, (3) training modalities, (4) benefits of repetition, (5) piercing it all together, (6) need for refresher training, and (7) provision of training before implant. Findings from this study suggest that SBML is a natural fit for the high-risk tasks needed to save VAD self-care. The 7 unique training-related themes derived from the qualitative data informed the design and development of a VAD SBML self-care curriculum.

Development of a Simulation-Based Mastery Learning Curriculum for Breaking Bad News

IntroductionPhysician communication impacts patient outcomes. However, communication skills, especially around difficult conversations, remain suboptimal, and there is no clear way to determine the validity of entrustment decisions. The aims of this study were to 1) describe the development of a simulation-based mastery learning (SBML) curriculum for breaking bad news (BBN) conversation skills and 2) set a defensible minimum passing standard (MPS) to ensure uniform skill acquisition among learners. InnovationAn SBML BBN curriculum was developed for fourth-year medical students. An assessment tool was created to evaluate the acquisition of skills involved in a BBN conversation. Pilot testing was completed to confirm improvement in skill acquisition and set the MPS. OutcomesA BBN assessment tool containing a 15-item checklist and six scaled items was developed. Students' checklist performance improved significantly at post-test compared to baseline (mean 65.33%, SD = 12.09% vs mean 88.67%, SD = 9.45%, P < 0.001). Students were also significantly more likely to have at least a score of 4 (on a five-point scale) for the six scaled questions at post-test. The MPS was set at 80%, requiring a score of 12 items on the checklist and at least 4 of 5 for each scaled item. Using the MPS, 30% of students would require additional training after post-testing. CommentsWe developed a SBML curriculum with a comprehensive assessment of BBN skills and a defensible competency standard. Future efforts will expand the mastery model to larger cohorts and assess the impact of rigorous education on patient care outcomes.

A Simulation-Based Mastery Learning Approach to Ventricular Assist Device Self-Management: The SimVAD Study

Background Care of ventricular assist devices (VADs) requires meticulous self-management skills by patients and caregivers to enhance competence and prevent adverse events. Simulation-based mastery learning (SBML) is a clinical educational model that uses mannequins to train healthcare providers to a high standard and has been shown to reduce patient complications. SBML has not been used to train patients or caregivers to mastery on required skills. This abstract describes the multidisciplinary development of an innovative patient-centered, VAD self-management SBML curriculum. Methods We completed multidisciplinary, semi-structured interviews with 11 physicians, 7 VAD coordinators, 16 patients and 12 caregivers to identify important factors in VAD self-management. From these responses, we created a device-specific SBML training curriculum. SBML curricula include pretests followed by deliberate practice (repetitive, focused skills practice with expert feedback) on a mannequin. After practice, participants must meet or exceed a minimum passing standard (MPS) on posttests. To set MPSs on posttests for our SBML curriculum, we convened an expert panel of 6 physicians, 6 VAD coordinators, 6 patients and 6 caregivers and used the Mastery Angoff standard setting method. Results Interviews identified critical factors for inclusion in the curriculum: critical learning content (e.g., medications, controller and dressing changes) and training modalities such as wearable simulators, educational videos, emergency checklists and refresher training sessions. We created educational videos on controller changes, dressing changes, and abnormal driveline sites. Emergency checklists were created to help reinforce knowledge of alarms and VAD-related items needed for travel. We created checklists for controller and dressing changes, and changing power sources for scoring pre and post skills exams. We also developed a post training written exam which included key knowledge components. The MPS, per expert panel, for the skills exams ranged from 94–99% items correct while the written exam was 96% items correct. One, three, and 12-month follow-up skills assessments were subsequently created. Conclusions This is the first patient-centered VAD self-management curriculum to be created utilizing a powerful educational modality (SBML) and multidisciplinary feedback. We are currently engaged in a pilot trial (SimVAD) of the SBML curriculum to assess knowledge, skills acquisition and health related outcomes. Download : Download high-res image (90KB) Download : Download full-size image

Simulation-based mastery learning significantly reduces gender differences on the Fundamentals of Endoscopic Surgery performance exam.

Analysis of the Fundamentals of Endoscopic Surgery (FES) performance exam showed higher scores for men than women. Gender differences have been reduced with task-specific practice. We assessed the effect of simulation-based mastery learning (SBML) on FES performance exam differences by gender. Forty-seven surgical trainees [29 men (m), 18 women (w)] completed a SBML curriculum and were assessed by FES. Fourteen trained on the GI Mentor 2, 18 on the Endoscopy Training System, and 15 using the Surgical Training for Endoscopic Proficiency curriculum. Performance of male and female trainees was compared. On the pre-training assessment, there were large differences between genders in FES pass rates (m 77%, w 15%, p < 0.001), total scores (m 69 ± 11, w 50 ± 12; p < 0.001), and in four of five FES sub-task scores (Navigation, m 73 ± 19, w 55 ± 22, p = 0.02; Loop reduction, m 34 ± 29, w 14 ± 22, p = 0.02; Retroflexion, m 81 ± 17, w 47 ± 27, p < 0.001; Targeting, m 89 ± 10, w 66 ± 23, p = 0.002). No differences were discernible post training (Pass rate, m 100%, w 94%, p = 0.4; Total score, m 77 ± 8, w 72 ± 12, p = 0.2; Navigation, m 91 ± 13, w 80 ± 13, p = 0.009; Loop reduction, m 49 ± 26, w 46 ± 36, p = 0.7; Retroflexion, m 82 ± 18, w 81 ± 15, p = 0.9; Targeting, m 92 ± 15, w 86 ± 12, p = 0.12). Time needed to complete curricula was not discernably different by gender (m 3.8 ± 1.7h, w 5.0 ± 2.6h, p = 0.17). Gender-based differences are nearly eliminated through task-specific SBML training. This lends further evidence to the validity argument for the FES performance exam as a measure of basic endoscopic skills.

Barriers and Facilitators to Central Venous Catheter Insertion: A Qualitative Study.

The aims of the study were to identify perceived barriers and facilitators to central venous catheter (CVC) insertion among healthcare providers and to understand the extent to which an existing Simulation-Based Mastery Learning (SBML) program may address barriers and leverage facilitators. Providers participating in a CVC insertion SBML train-the-trainer program, in addition to intensive care unit nurse managers, were purposively sampled from Veterans Administration Medical Centers located in geographically diverse areas. We conducted semistructured interviews to assess perceptions of barriers and facilitators to CVC insertion. Deidentified transcripts were analyzed using a grounded theory approach and the constant comparative method. We subsequently mapped identified barriers and facilitators to our SBML curriculum to determine whether or not the curriculum addresses these factors. We interviewed 28 providers at six Veterans Administration Medical Centers, identifying the following five overarching factors of perceived barriers to CVC insertion: (1) equipment, (2) personnel/staff, (3) setting or organizational context, (4) patient or provider, and (5) time-related barriers. Three overarching factors of facilitators emerged: (1) equipment, (2) personnel, and (3) setting or organizational context facilitators. The SBML curriculum seems to address most identified barriers, while leveraging many facilitators; building on the commonly identified facilitator of nursing staff contribution by expanding the curriculum to explicitly include nurse involvement could improve team efficiency and organizational culture of safety. Many identified facilitators (e.g., ability to use ultrasound, personnel confidence/competence) were also identified as barriers. Evidence-based SBML programs have the potential to amplify these facilitators while addressing the barriers by providing an opportunity to practice and master CVC insertion skills.

Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the Endoscopy Training System

BackgroundThe Fundamentals of Endoscopic Surgery (FES) exam is required for American Board of Surgery certification. The purpose of this study was to develop performance standards for a simulation-based mastery learning (SBML) curriculum for the FES performance exam using the Endoscopy Training System (ETS). MethodsExperienced endoscopists from multiple institutions and specialties performed each ETS task (scope manipulation (SM), tool targeting (TT), retroflexion (RF), loop management (LM), and mucosal inspection (MI)) with scores used to develop performance standards for a SBML training curriculum. Trainees completed the curriculum to determine feasibility, and effect on FES performance. ResultsTask specific training standards were determined (SM-121sec, TT-243sec, RF-159sec, LM-261sec, MI-180-480sec, 7 polyps). Trainees required 29.5 ± 3.7 training trials over 2.75 ± 0.5 training sessions to complete the SBML curriculum. Despite high baseline FES performance, scores improved (pre 73.4 ± 7, post 78.1 ± 5.2; effect size = 0.76, p > 0.1), but this was not statistically discernable. ConclusionsThis SBML curriculum was feasible and improved FES scores in a group of high performers. This curriculum should be applied to novice endoscopists to determine effectiveness for FES exam preparation.

Simulation-based mastery learning for endoscopy using the endoscopy training system: a strategy to improve endoscopic skills and prepare for the fundamentals of endoscopic surgery (FES) manual skills exam.

The fundamentals of endoscopic surgery (FES) program has considerable validity evidence for its use in measuring the knowledge, skills, and abilities required for competency in endoscopy. Beginning in 2018, the American Board of Surgery will require all candidates to have taken and passed the written and performance exams in the FES program. Recent work has shown that the current ACGME/ABS required case volume may not be enough to ensure trainees pass the FES skills exam. The aim of this study was to investigate the feasibility of a simulation-based mastery-learning curriculum delivered on a novel physical simulation platform to prepare trainees to pass the FES manual skills exam. The newly developed endoscopy training system (ETS) was used as the training platform. Seventeen PGY 1 (10) and PGY 2 (7) general surgery residents completed a pre-training assessment consisting of all 5 FES tasks on the GI Mentor II. Subjects then trained to previously determined expert performance benchmarks on each of 5 ETS tasks. Once training benchmarks were reached for all tasks, a post-training assessment was performed with all 5 FES tasks. Two subjects were lost to follow-up and never returned for training or post-training assessment. One additional subject failed to complete any portion of the curriculum, but did return for post-training assessment. The group had minimal endoscopy experience (median 0, range 0-67) and minimal prior simulation experience. Three trainees (17.6%) achieved a passing score on the pre-training FES assessment. Training consisted of an average of 48±26 repetitions on the ETS platform distributed over 5.1±2 training sessions. Seventy-one percent achieved proficiency on all 5 ETS tasks. There was dramatic improvement demonstrated on the mean post-training FES assessment when compared to pre-training (74.0±8 vs. 50.4±16, p<0.0001, effect size=2.4). The number of ETS tasks trained to proficiency correlated moderately with the score on the post-training assessment (r=0.57, p=0.028). Fourteen (100%) subjects who trained to proficiency on at least one ETS task passed the post-training FES manual skills exam. This simulation-based mastery learning curriculum using the ETS is feasible for training novices and allows for the acquisition of the technical skills required to pass the FES manual skills exam. This curriculum should be strongly considered by programs wishing to ensure that trainees are prepared for the FES exam.

Developing a Simulation-Based Mastery Learning Curriculum

Developing a Simulation-Based Mastery Learning Curriculum

Developing a Simulation-Based Mastery Learning Curriculum: Lessons From 11 Years of Advanced Cardiac Life Support.

Curriculum development in medical education should follow a planned, systematic approach fitted to the needs and conditions of a local institutional environment and its learners. This article describes the development and maintenance of a simulation-based medical education curriculum on advanced cardiac life support skills and its transformation to a mastery learning program. Curriculum development used the Kern 6-step model involving problem identification and general needs assessment, targeted needs assessment, goals and objectives, educational strategies, implementation, and evaluation and feedback. Curriculum maintenance and enhancement and dissemination are also addressed. Transformation of the simulation-based medical education curriculum to a mastery learning program was accomplished after a 2-year phase-in trial. A series of studies spanning 11 years was performed to adjust the curriculum, improve checklist outcome measures, and evaluate curriculum effects as learning outcomes among internal medicine residents and improved patient care practices. We anticipate wide adoption of the mastery learning model for skill and knowledge acquisition and maintenance in medical education settings.

Board #209 - Program Innovation Dissemination of a Simulation-based Mastery Learning Curriculum for Central Venous Catheter Insertion at Veterans Affairs Hospitals (Submission #10180)

Board #209 - Program Innovation Dissemination of a Simulation-based Mastery Learning Curriculum for Central Venous Catheter Insertion at Veterans Affairs Hospitals (Submission #10180)

Dissemination of a simulation-based mastery learning intervention reduces central line-associated bloodstream infections

BackgroundApproximately 41 000 central line-associated bloodstream infections (CLABSI) occur annually in US hospitals. We previously developed a simulation-based mastery learning (SBML) curriculum in central venous catheter (CVC) insertion that significantly...

Board 305 - Research Abstract Dissemination of a Simulation-Based Mastery Learning Intervention Reduces Central Line-Associated Bloodstream Infections (Submission #202)

Introduction/Background Approximately 41,000 central line-associated bloodstream infections (CLABSI) occur yearly in U.S. hospitals.1 Despite strategies such as checklists, many hospitals have not been able to reduce CLABSI rates.2 We previously developed a simulation-based mastery learning (SBML) curriculum in central venous catheter (CVC) insertion that significantly reduced CLABSI rates.3 The aim of the current study was to measure the effect of dissemination and implementation of the SBML curriculum developed at a tertiary care medical center on trainee skills and CLABSI rates at a community hospital. Methods We performed a cohort study of all internal medicine and emergency medicine residents before their intensive care unit (ICU) rotation at Mercy Hospital (an academic community hospital) from September 2010 to May 2012. Residents underwent a CVC SBML curriculum before their ICU rotation. We evaluated residents simulated internal jugular (IJ) and subclavian (SC) CVC insertion skills before and after the educational intervention using a 29 item checklist. All residents were required to meet or exceed a minimum passing score at post-test before completion of training. Subsequently, we evaluated the effects of the SBML training program on CLABSI rates at Mercy hospital. We compared CLABSI rates from October 2008 to August 2010 (pre-intervention) to September 2010 to May 2012 (post-intervention). Results The mean IJ pretest score was 35.5% (10.29/29, SD = 8.30) compared to a post-test mean of 93.0% (26.96/29, SD = 1.50; P < 0.001). The mean SC pretest score was 23.0% (6.68/29; SD = 9.58) and increased to 96.1% (27.88/29, SD = 1.41) at post-test (P < 0.001). There were 3.82 infections per 1000 catheter-days (20 infections in 5235 catheter-days) in the ICU in the 23 months before the educational intervention. During the 21 months after the intervention there were 1.29 infections per 1000 catheter-days (6 infections in 4670 catheter days). Patients in the ICU had significantly fewer CLABSIs after the SBML intervention (P = 0.019). The incidence rate ratio was 0.257 (95% CI, 0.089-0.742) after controlling for APACHE III scores. Conclusion This study demonstrates successful dissemination and implementation of a CVC SBML curriculum at a community academic hospital. The intervention significantly improved resident CVC insertion skills and reduced CLABSI rates. This collaboration illustrates how medical educators can use evidence-based training Methods to improve patient care.

Simulation-Based Mastery Learning Improves Patient Outcomes in Laparoscopic Inguinal Hernia Repair

To evaluate a mastery learning, simulation-based curriculum for laparoscopic, totally extraperitoneal (TEP) inguinal hernia repair. Clinically relevant benefits from improvements in operative performance, time, and errors after simulation-based training are not clearly established. After performing a baseline TEP in the OR, general surgery residents randomized to mastery learning (ML) or standard practice (SP) were reassessed during subsequent TEPs. The ML curriculum involved Web-based modules followed by training on a TEP simulator until expert performance was achieved. Operative time, performance, and patient outcomes adjusted for staff, resident participation, difficulty of repair, PGY-level, and patient comorbidities were compared between groups with mixed effects-ANOVA and generalized linear models. Fifty residents (PGY1-5) performed 219 TEP repairs on 146 patients. Baseline operative time, performance, and demographics were similar between groups. To achieve mastery, ML-residents (n = 26) required a median of 16 (range 7-27) simulated repairs. After training, TEPs performed by ML-residents were faster than those by SP-residents, with time corrected for participation (mean ± SD, 34 ± 8 minutes vs. 48 ± 14 minutes; difference -13; 95%CI, -18 to -8; P < 0.001). Operative performance scores (GOALS, scale 6-30) were better for ML residents (21.9 ± 2.8 vs. 18.3 ± 3.8; P = 0.001). Intraoperative complications (peritoneal tear, procedure conversion), postoperative complications (urinary retention, seroma), and need for overnight stay were less likely in the ML group (adjusted odds ratios 0.14, 0.04, and 0, respectively; all P < 0.05). A simulation-based ML curriculum decreased operative time, improved trainee performance, and decreased intra- and postoperative complications and overnight stays after laparoscopic TEP inguinal hernia repair. ClinicalTrials.gov Identifier: NCT01085500.