Purpose: In response to the twin pandemics of COVID-19 and racial inequity, our health professions have awakened to the impact of structural racism on health care. While the current generation of trainees is facile with cultural humility, the generation of faculty lags. 1,2 Faculty development programs aimed at closing this gap are essential. 3 Most studies of cultural competency training focus on outcomes limited to practitioner knowledge acquisition and attitudes, not skills. 4 In implementing a year-long faculty development program in diversity, equity, and inclusion (DEI), we asked ourselves: Is it possible to assess faculty’s skills of antiracism and allyship to engage learners using an OSCE (observed structured clinical examination)? Method: Our faculty development in DEI, grounded in longitudinal community of practice, uses appreciative inquiry, critical reflection, and skill building. We designed a mid-program OSCE and OSTE (observed structured teaching examination) to assess skill acquisition. In the OSCE, participants conduct an advising session with a trainee underrepresented in medicine who is experiencing stereotype threat. In the OSTE, participants facilitate a teaching session on COVID-19 vaccine hesitancy. Fourth-year students, trained as standardized learners, completed behavioral checklists. Faculty participants self-assessed using the same behavioral checklists. Faculty development leaders served as observers and led global feedback discussions with the faculty and learners who then completed surveys for reflection. Results: In the advising OSCE, faculty demonstration of the 8 core skills ranged from 17% to 100% (average 63%). While 17% of the faculty named stereotype threat, 100% implemented one strategy to mitigate it. While 50% elicited a personal narrative from the learner, 67% were able to disclose their own “location of self.” While 58% inquired about experiences of bias, 83% responded with empathy after hearing them. The standardized learner reported comfort with 58% of the faculty as an advisor. In the vaccine hesitancy OSTE, faculty demonstration of the 12 core skills ranged from 33% to 100% (average 78%). Only 33% named their “location of self” yet the standardized learners reported comfort with 100% of the faculty as small-group leaders. Ninety-two percent of the faculty taught the sociopolitical construct of race and connected historical racism to vaccine hesitancy. Seventy-five percent of the faculty used one core skill of establishing psychological safety, and 67% created ground rules distinguishing safe from brave spaces. The kappa coefficient ranged from 0.26 to 0.88, depending on the learners involved. One hundred percent of the faculty and standardized learners agreed that the OSCE/OSTE was valuable, the stations were relevant and appropriate, they were able to identify strengths and weaknesses, and the experience made them feel positive about efforts to improve the learning environment. One hundred percent agreed that logistics went smoothly. Discussion: We discovered that OSCE/OSTEs for the assessment of DEI skill acquisition are not only feasible but also valuable, to faculty participants, program leaders, and standardized learners. We identified gaps in the curriculum and a need for better training for interrater reliability. We also identified potential future facilitators. Finally, our student participants were impressed with the quality of the curriculum and assessment. This message, organically shared among all students, was an unintended benefit to the school. Significance: To our knowledge, this OSCE/OSTE is the first of its kind to formatively assess DEI skills. While success on a behavioral checklist is still a proxy for how faculty behave in the actual learning environment, it goes a long way toward identifying skill acquisition. We plan to refine this assessment mid-program and implement an end-of-program OSCE/OSTE for skills of apology, upstanding and leveraging privilege. We will make our materials available as a free open-access medical education resource. Acknowledgments: The authors wish to thank Treisha Green and Kelly Conlon for their assistance with this project, as well as all the ZSOM students who volunteered their time to play standardized learners.
Read full abstract