Debriefing Assessment For Simulation In Healthcare Research Articles

Training and evaluating simulation debriefers in low-resource settings: lessons learned from Bihar, India

BackgroundTo develop effective and sustainable simulation training programs in low-resource settings, it is critical that facilitators are thoroughly trained in debriefing, a critical component of simulation learning. However, large knowledge gaps exist regarding the best way to train and evaluate debrief facilitators in low-resource settings.MethodsUsing a mixed methods approach, this study explored the feasibility of evaluating the debriefing skills of nurse mentors in Bihar, India. Videos of obstetric and neonatal post-simulation debriefs were assessed using two known tools: the Center for Advanced Pediatric and Perinatal Education (CAPE) tool and Debriefing Assessment for Simulation in Healthcare (DASH). Video data was used to evaluate interrater reliability and changes in debriefing performance over time. Additionally, twenty semi-structured interviews with nurse mentors explored perceived barriers and enablers of debriefing in Bihar.ResultsA total of 73 debriefing videos, averaging 18 min each, were analyzed by two raters. The CAPE tool demonstrated higher interrater reliability than the DASH; 13 of 16 CAPE indicators and two of six DASH indicators were judged reliable (ICC > 0.6 or kappa > 0.40). All indicators remained stable or improved over time. The number of ‘instructors questions,’ the amount of ‘trainee responses,’ and the ability to ‘organize the debrief’ improved significantly over time (p < 0.01, p < 0.01, p = 0.04). Barriers included fear of making mistakes, time constraints, and technical challenges. Enablers included creating a safe learning environment, using contextually appropriate debriefing strategies, and team building. Overall, nurse mentors believed that debriefing was a vital aspect of simulation-based training.ConclusionSimulation debriefing and evaluation was feasible among nurse mentors in Bihar. Results demonstrated that the CAPE demonstrated higher interrater reliability than the DASH and that nurse mentors were able to maintain or improve their debriefing skills overtime. Further, debriefing was considered to be critical to the success of the simulation training. However, fear of making mistakes and logistical challenges must be addressed to maximize learning. Teamwork, adaptability, and building a safe learning environment enhanced the quality enhanced the quality of simulation-based training, which could ultimately help to improve maternal and neonatal health outcomes in Bihar.

Spinal Epidural Abscess.

Spinal Epidural Abscess.

The impact of a high fidelity simulation-based debriefing course on the Debriefing Assessment for Simulation in Healthcare (DASH)© score of novice instructors.

Introduction:Experiential learning, followed by debriefing, is at the heart of Simulation-Based Medical Education (SBME) and has been proven effective to help master several medical skills. We investigated the impact of an educational intervention, based on high-fidelity SBME, on the debriefing competence of novice simulation instructors. Methods:This is a prospective, randomized, quasi-experimental, pre- and post-test study. Sixty physicians without prior formal debriefing expertise attended a 5-day SBME seminar targeted on debriefing. Prior to the start of the seminar, 15 randomly chosen participants had to debrief a spaghetti and tape team exercise. Thereafter, the members of each team assessed their debriefer’s performance using the Debriefing Assessment for Simulation in Healthcare (DASH)© score. The debriefing seminar that followed (intervention) consisted of 5 days of teaching that included theoretical and simulation training. Each scenario was followed by a Debriefing of the Debriefing (DOD) session conducted by the expert instructor. At the end of the course, 15 randomly chosen debriefers had to debrief a second tower building exercise and were re-evaluated with the DASH score by their respective team members. The Wilcoxon signed-rank test was used to compare pre- and post-test scores. Statistical tests were performed using GraphPad Prism 6.0c for Mac. Results: A significant improvement in all items of the DASH score was noted following the seminar. The debriefers significantly improved their performance with regard to “maintaining an engaging learning environment” (Median [IQR]) (4[3-5] after the pre-test vs. 5.5[5-6] after the post-test, p<0.001); “structuring the debriefing in an organized way” (5[4-5] after the pre-test vs. 5[5-6] after the post-test, p=0.002); “provoking engaging discussion” (4[3-5.75] after the pre-test vs. 6[5-6] after the post-test, p<0.001); “identifying and exploring performance gaps” (5[4-6] after the pre-test vs. 6[5-6] after the post-test, p=0.014); and “helping trainees to achieve and sustain good future performance” (4[3-5] after the pre-test vs. 6[5-6] after the post-test, p<0.001). Conclusion:A simulation-based debriefing course, based mainly on DOD sessions, allowed novice simulation instructors to improve their overall debriefing skills including, more specifically, the ability to foster engagement in discussions and maintain an engaging learning environment.

A novel simulator model and standardized assessment tools for fine needle aspiration cytology training.

Fine needle aspiration (FNA) is an invaluable diagnostic procedure for evaluation of lesions; however, acquisition of diagnostic material is dependent on the skill of the practitioner. We report a novel patient simulator for teaching the FNA procedure and structured assessment tools for educators and learners. We created a novel simulator model for FNA training, employed a standardized teaching module, and assessed procedure utility in medical students. Groups of students completed training using a commercial version of the model, and underwent structured evaluation using an Objective Structured Assessment of Technical Skills (OSATS) form, and the Debriefing Assessment for Simulation in Healthcare (DASH) tool. In the initial phase, 178 students rated the training workshop between valuable and essential (4.2 on a 5-point Likert scale). In the second phase, for students evaluated with the OSATS form, the mean overall score was 33 out of 50 (range 26-43). The areas of weakness for the participants were: (a) compression after the FNA procedure, (b) completion of the informed consent, and (c) correct explanation of the procedure to the patient. For the group of students that completed the DASH questionnaire, the results were: 6.2 (assessment by students) and 6.7 (assessment by instructor) out of a maximum of 7. A realistic simulation model, in combination with a standardized training program with formal assessment methods is a valuable tool to teach FNA. We here describe a process for teaching the FNA procedure to interested educators and learners.

Implementing Standardized Patients Within Simulation in a Nurse Practitioner Program

BackgroundNurse practitioner (NP) faculty need to think strategically about how to best implement simulation using standardized patients (SPs) for primary care NP courses. This pilot study sought to identify barriers and strategies to facilitate the use of SPs in a NP program. MethodNP students participated in an ambulatory care simulation using a SP. Students completed the Debriefing Assessment for Simulation in Healthcare (DASH)© student version, short form, after the simulation. ResultsStudents rated the simulation debriefing experience as effective in helping them improve or sustain good performance. ConclusionAlthough SPs portrayed a realistic ambulatory care environment, barriers are identified along with suggestions to overcome these barriers.

Board 336 - Research Abstract Inexperienced Debriefer Use of the DASH Tool to Assess Peer Debrief Performance (Submission #1089)

Introduction/Background The DASH© tool is a debriefing assessment scale that has been developed by the Harvard Medical Simulation Centre for use by those trained in its use, and is a valid and reliable measure when used in such a fashion. 1 There is no data on its use by the inexperienced and untrained debriefer. The tool requires the assessor to assess across six domains covering all aspects of the debrief, with each point scored on a seven point Likert-like scale. Methods Five inexperienced participants in a faculty development programme were invited to each debrief one pre-recorded simulation scenario. The other four participants observed this debriefing and were asked to complete a DASH evaluation of that debriefing with only a short introduction to the use of the tool. These responses were recorded anonymously and descriptive analysis of the responses performed and non-parametric statistical analysis to establish any significant difference between assesees or domains. Results Twelve completed assessments were returned out of a total of 16 possible. The mean response across all questions for all completed responses was 4.48 (sd 0.45). Individual domain mean responses were all in the range 3.92 to 4.92. Standard deviation for responses across the data set indicated little spread (range 0.65 to 1.24). Kruskal-Wallis analysis demonstrated no significant difference between assesses for any domain of the DASH tool. Analysis of response to each domain similarly revealed a narrow distribution around the median value, with only one response falling out with the range three to six. Again, there was no significant difference across domains for any response. Conclusion These Results suggest that the inexperienced debriefer when using the DASH tool may default towards the median value in the seven-point scale, possibly with a tendency towards more positive response. There appeared to be a hesitancy to use the extremes of the scale. The inexperienced debriefer may not have the confidence or experience to accurately assess and feedback on others performance in debrief and further work should be done to examine this more closely. Reference 1. Brett-Fleegler M, Rudolph JW, Eppich WJ, Fleegler E, Cheng A, Simon RS. Debriefing Assessment for Simulation in Healthcare (DASH): Assessment of teh Reliability of a Debriefing Instrument. 2009;4(4):240–325. Disclosures None.

Medical Students' Perception of Residents as Teachers: Comparing Effectiveness of Residents and Faculty During Simulation Debriefings

Simulation can enhance undergraduate medical education. However, the number of faculty facilitators needed for observation and debriefing can limit its use with medical students. The goal of this study was to compare the effectiveness of emergency medicine (EM) residents with that of EM faculty in facilitating postcase debriefings. The EM clerkship at Indiana University School of Medicine requires medical students to complete one 2-hour mannequin-based simulation session. Groups of 5 to 6 students participated in 3 different simulation cases immediately followed by debriefings. Debriefings were led by either an EM faculty volunteer or EM resident volunteer. The Debriefing Assessment for Simulation in Healthcare (DASH) participant form was completed by students to evaluate each individual providing the debriefing. In total, 273 DASH forms were completed (132 EM faculty evaluations and 141 EM resident evaluations) for 7 faculty members and 9 residents providing the debriefing sessions. The mean total faculty DASH score was 32.42 and mean total resident DASH score was 32.09 out of a possible 35. There were no statistically significant differences between faculty and resident scores overall (P = .36) or by case type (P trauma = .11, P medical = .19, P pediatrics = .48). EM residents were perceived to be as effective as EM faculty in debriefing medical students in a mannequin-based simulation experience. The use of residents to observe and debrief students may allow additional simulations to be incorporated into undergraduate curricula and provide valuable teaching opportunities for residents.

Debriefing Assessment for Simulation in Healthcare

This study examined the reliability of the scores of an assessment instrument, the Debriefing Assessment for Simulation in Healthcare (DASH), in evaluating the quality of health care simulation debriefings. The secondary objective was to evaluate whether the instrument's scores demonstrate evidence of validity. Two aspects of reliability were examined, interrater reliability and internal consistency. To assess interrater reliability, intraclass correlations were calculated for 114 simulation instructors enrolled in webinar training courses in the use of the DASH. The instructors reviewed a series of 3 standardized debriefing sessions. To assess internal consistency, Cronbach α was calculated for this cohort. Finally, 1 measure of validity was examined by comparing the scores across 3 debriefings of different quality. Intraclass correlation coefficients for the individual elements were predominantly greater than 0.6. The overall intraclass correlation coefficient for the combined elements was 0.74. Cronbach α was 0.89 across the webinar raters. There were statistically significant differences among the ratings for the 3 standardized debriefings (P < 0.001). The DASH scores showed evidence of good reliability and preliminary evidence of validity. Additional work will be needed to assess the generalizability of the DASH based on the psychometrics of DASH data from other settings.

Assessing and improving debriefing: A workshop using the Debriefing Assessment for Simulation in Healthcare (DASH)

Based on extensive research and experience a behaviorally anchored rating scale for assessing debriefings, the Debriefing Assessment for Simulation in Healthcare (DASH) is available for widespread use. Participants will be given the DASH and trained in its use. The instrument assesses eight crucial elements for conducting an effective debriefing. The workshop will provide participants a chance to observe debriefings, use the DASH to assess the debriefings, and discuss the ratings. First, the DASH will be given to participants followed by a discussion of the theory behind it and directions for its use. Next, participants will engage in an active learning cycle in which they will observe a scenario, practice using the DASH on the subsequent debriefing, compare ratings and discuss differences. This practice cycle (observation – ratings – discussion) will be repeated three times; each time a different debriefing style will be observed.