ICRE Top Research Abstracts

Abstract

Editor's Note: The following are the Top Research in Residency Education abstracts selected by the Journal of Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada for the 2021 International Conference on Residency Education (ICRE).Introduction: Faculty assessment burden is an anticipated effect of competency-based medical education (CBME), but what is the workload for residents? Additional faculty workload is deemed necessary, because the purpose of CBME is to provide a learner-centered education that supports the development of competent physicians and surgeons. However, good intentions can have unintended effects. We need to understand assessment workload in CBME from the perspective of residents.Methods: This study investigates how CBME theory translates to practice in operative and perioperative postgraduate training programs in Canada. The project employs constructivist grounded theory to develop explanatory, contextual models of CBME in social practice. Nineteen residents from 5 training programs in 5 institutions participated in the study through semi-structured interviews averaging 1 hour in length. Questions explored the resident experience of CBME implementation. Themes of a theory to practice “disconnect” concerning intentions for self-regulated learning and formative feedback were identified initially; subsequent interviews explored how these disconnects were occurring through the method of theoretical sampling. Two members of the research team independently coded the data and discussed relationships between concepts with the other members of the team to reach analytic consensus on the results through constant comparison.Results: The experience of assessment burden was strikingly consistent for residents. Rather than being informed and self-directed in their learning, residents described being driven to manage acquisition of entrustable professional activities (EPAs) in a way that made their progression stressful and opaque. While personalities, phase of implementation, and local training cultures played a role in mediating stress, the workload of assessment was generated more by external drivers such as EPA standards and technology interfaces.Conclusions: Resident well-being is at risk through unanticipated assessment burden in managing EPA progression. This research offers critical insight into how this unintended effect of CBME operates and how it can be mitigated.M. Ott, R. Pack, S. Cristancho, M. Chin, J. VanKoughnettWestern University, London, ON, CanadaIntroduction: Workplace-based assessment (WBA) is key to a competency-based assessment strategy. Concomitantly with our program's launch of Competence by Design, we developed a new formative WBA, the Anesthesia Clinical Encounter Assessment (ACEA), to assess readiness for independence (ie, entrustability) for competencies essential for perioperative patient care. This study aimed to examine validity evidence of the ACEA during postgraduate anesthesiology training.Methods: The ACEA comprises an 8-item global rating scale (GRS), an 8-item checklist, an overall independence rating, and case details. ACEA data were extracted for the University of Toronto anesthesia residents from July 2017 to January 2020. Validity evidence was generated from sources based on the unified theory of validity, including internal structure, relations with other variables, and consequences.Results: We analyzed 8536 assessments for 137 residents completed by 341 assessors. From generalizability analysis, 10 observations (2 assessments each from 5 assessors) were sufficient to achieve the reliability threshold of 0.70 for in-training evaluations. A mean GRS score of 3.65 out of 5 provided optimal sensitivity (94%) and specificity (91%) for determining competency on ROC analysis. Test-retest reliability was high (ICC = 0.81) for matched assessments within 14 days of each other. Mean GRS scores differed significantly between residents based on their training level (P < .0001) and correlated highly with overall independence (0.91, P < .001). The internal consistency of the GRS (a = 0.96) was excellent.Conclusions: This study provides evidence supporting validity of the ACEA for assessing the competence of residents performing perioperative care and supports its use in competency-based anesthesiology training.A. Kealey1, F. Alam1, L. Bahrey2, C. Matava3, G. McCreath3, C. Walsh31Sunnybrook Health Sciences Centre, Toronto, ON, Canada; 2Toronto General Hospital, Toronto, ON, Canada; 3Hospital for Sick Children, Toronto, ON, CanadaIntroduction: Entrustment, a central construct in competency-based medical education (CBME), is operationalized in the assessment of entrustable professional activities (EPAs). While EPA assessment is foundational in many CBME systems, research has yet to clarify how supervisors form judgments when assessing the same EPA in both workplace- and simulation-based settings. We aimed to explore the features supervisors report as influencing their entrustment decisions across these assessment settings.Methods: We conducted an interview-based, constructivist grounded theory–informed study involving gastroenterology supervisors and residents at the University of Toronto and the University of Calgary. Supervisors completed separate EPA assessments of a resident's endoscopic polypectomy (a relevant specialty-specific EPA) performance in both workplace and simulation-based settings. Supervisors were interviewed after each encounter to explore how they made their entrustment decisions within and across settings. Transcribed interview data were coded iteratively using constant comparison to generate themes.Results: Based on 20 interviews with 10 supervisors, we found that participants: (1) held multiple meanings of entrustment, both within and across participants; (2) expressed variability in how they justified their entrustment decisions, the related narrative, and numerical EPA assessment scoring; (3) held certain criteria for making entrustment decisions “comfortably” (eg, task authenticity, task-related variability, and opportunity to assess trainee response to unexpected events); and (4) perceived a relative freedom when using simulation to make entrustment decisions due to the absence of a real patient.Conclusions: We found that participants spoke about and defined entrustment in a variety of ways. That variety appeared to lead to variability in how supervisors judged entrustment, both within and across participants and assessment settings. These observed rater idiosyncrasies suggest residency programs cannot assume equivalence of EPA data from different assessment settings. Furthermore, CBME faculty development likely needs to attend to the criteria that supervisors report needing to comfortably make entrustment decisions.T. Jeyalingam1, C. Walsh1, W. Tavares1, M. Mylopoulos1, K. Hodwitz1, L. Liu2, S. Heitman2, R. Brydges11University of Toronto, Toronto, ON, Canada; 2University of Calgary, Calgary, AB, CanadaEditor's Note: The following are the Top Resident Research abstracts selected by the Journal of Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada for the 2021 International Conference on Residency Education (ICRE).Introduction: Racial diversity is vital in promoting the development of culturally competent physicians. To help diversify the applicant pool, many medical schools actively recruit applicants through underrepresented minority (URM) application streams. However, many URM medical students experience acts of marginalization throughout their training, and it is unclear how matriculants from URM streams are perceived by their peers. This study uses online medical discussion forums to provide insight into how URM streams across North America are perceived by both URM and non-URM applicants.Methods: A total of 840 posts from 13 discussion threads in Premed101 (Canadian) and Student Doctor Network (American) discussion forums were analyzed. Participants in the forum included undergraduate students, applicants to medical school, and current medical students from URM and non-URM groups. Inductive content analysis was used to develop a data-driven coding scheme from which several common themes were identified.Results: Despite an overall appreciation for the benefits of a diverse workforce, participants in the forums engaged in heated discussion surrounding URM streams in North America. Our analysis identified prominent perceptions that students admitted from URM streams are of lower quality, that there is a broken admissions process with fear of social change, and that the influence of socioeconomic status is underappreciated by medical schools.Conclusions: Online discussion forums are a novel platform to provide insight into the perceptions surrounding URM medical admission streams. Our study identifies both barriers and enticing factors that influence application to these programs. Ultimately, we highlight prominent misconceptions against which actionable measures should be taken to reduce marginalization against students admitted through these streams.H. Barootes1, A. Cong-anh Huynh1, M. Maracle1, A. Istl2, P. Wang3, A. Kirpalani31Western University, London, ON, Canada; 2Johns Hopkins University, Baltimore, MD, USA; 3London Health Sciences Centre, London, ON, CanadaIntroduction: Competency-based medical education strives to address societal needs and be learner-centered through its focus on outcomes and observable activities that inform progression of trainee competence. Assessment of competency remains challenging in residency and continues to largely focus on physicians' inferred judgments from case presentations rather than direct observation in the workplace. Seeking patients' perspectives on fundamental competencies could be an influential addition to workplace-based assessment methods. However, the role of patients as assessors deserves additional attention as to how they can inform our feedback practices and contribute to competency decisions. We aimed to explore the evidence for patient involvement using psychometric tools in resident assessment.Methods: Guided by Arskey and O'Malley's framework for scoping reviews, we searched 3 databases (MEDLINE, PubMed, and Embase) prior to November 2019, and updated in July 2020 and February 2021. Two authors independently assessed records for eligibility and included empirical studies of all designs that examined patient involvement in the assessment of residents.Results: We identified 821 records with 41 having met all eligibility criteria. A range of specialties were represented in our study. Patients were primarily included across ambulatory (22 of 41, 53.7%) and inpatient (13 of 41, 31.7%) settings. One-third of studies included patients as a component of a broader assessment approach (eg, multisource [360°] feedback with patient engagement [14 of 41, 34.1%]). The Communication Assessment Tool was the most used instrument (9 of 41, 22.0%) to collect patient feedback. Patients generally provided high ratings and comments on the observed professional behaviors and communication skills in comparison to physicians who focused on medical expertise.Conclusions: Our review suggests that involving patients in resident assessment is feasible and may offer unique insights that are not captured in assessments completed by physicians or other providers. How patients can help confirm or advance judgements on competence attainment remains uncertain but understudied.C. Gonsalves1, M. Gupta1, T. A. Horsley2, R. Khalife1, Y. Park3, J. Riddle4, A. Tekian41University of Ottawa, Ottawa, ON, Canada; 2Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada; 3Harvard Medical School, Boston, MA, USA; 4University of Illinois at Chicago, Chicago, IL, USAIntroduction: Arising from the impacts of historic and ongoing colonialism, Indigenous Peoples in Canada are underrepresented in medicine, and this is accentuated in obstetrics and gynecology (OB/GYN) and other surgical specialties. In an effort to decolonize the admissions process and address the need for substantive equality in postgraduate training, the University of Alberta OB/GYN residency program has developed an Indigenous Admissions Pathway (IAP). This abstract describes the development of the IAP and describes our evaluation of the postgraduate training goals and perspectives of self-identified Indigenous medical students.Methods: Indigenous and allied faculty, residents, community members, and Elders created the application and review process for the IAP. Self-identified Indigenous students at a Canadian medical school were invited to participate in an electronic survey. Analysis included descriptive statistics and a thematic analysis of open-ended questions.Results: The IAP is in its second year. Applicants to the program apply in the usual manner and indicate their application to the IAP through a separate letter of intent. A panel interview including Indigenous faculty, residents, and Elders is held in parallel to the residency program interviews. Thirty-six participants responded to the survey. Eighty-one percent of participants felt that an IAP would influence their choice of residency training program, and 75% would choose this option when applying to a residency program. All participants thought that an IAP would have a positive impact on the delivery of care for Indigenous patients.Conclusions: This study supports the ongoing use of the OB/GYN IAP. An IAP must be accompanied by a robust program of Indigenous professional development and other effective, community-driven initiatives to decolonize postgraduate medical education. This study will be used to improve the IAP with the ultimate goal of increasing Indigenous representation in OB/GYN and improving access to culturally safe care.K. Black, C. Felske-Durksen, C. Flood, M. Hyakutake, T. Kemble, M. Manniapik, H. McKennitt, B. Schroeder, R. RichUniversity of Alberta, Edmonton, AB, CanadaIntroduction: Patient death is an inevitability of medical training. Subsequent distress, decreased empathy, and worse learning outcomes have been reported among physicians and residents. While debriefing provides space for reflection, which promotes a supportive culture, this infrequently occurs. Early trainees often feel underprepared to manage death. We aimed to ascertain the impacts of patient death, debriefing opportunities, and coping strategies employed by residents at McMaster University.Methods: Trainees across various residency programs who completed an internal medicine rotation at McMaster University were invited to participate. Semi-structured interviews were conducted to understand circumstances, emotional responses, support, coping mechanisms, and preparedness regarding the patient death experience. Interviews were transcribed and coded to identify emerging themes using thematic analysis and constructivist grounded theory.Results: At the time of submission, 10 interviews were conducted and 18 participants recruited. Three main themes were categorized: 1–patient death circumstances; 2–personal and professional impact; and 3–trainee support. Pronouncing death, communicating with families, and unexpected/unknown deaths were common challenges. Feelings of guilt, helplessness, regret, and grief often followed events, amplified by lack of debriefs. Perceived medical culture, power imbalances between staff and trainees, and fear of appearing unprofessional contributed to emotional consequences, which included difficulties sleeping, intrusive thoughts, and emotional distancing in subsequent deaths. Respondents universally felt underprepared for the experience. Some residents were aware of program supports, although none accessed these services. While these experiences are congruent with effects of psychological trauma, they were consistently normalized by trainees.Conclusions: Patient death in medical training can be traumatic for learners and may perpetuate loss of empathy, changes to practice, and residual emotional effects. These experiences are normalized by the medical environment, culture, and the residents themselves. Further focus is needed to better prepare trainees for this phenomenon and examine the culture in which physicians operate.W. Ye1, C. Griffin1, I. Sverdlichenko2, D. Brandt Vegas11McMaster University, Hamilton, ON, Canada; 2University of Toronto, Toronto, ON, CanadaIntroduction: Social Pediatrics is the newest rotation included in the General Pediatrics Residency Program at the University of Alberta. Evaluation involves a written reflective assignment identifying assets and disparities that have influenced the health of a child encountered on the rotation. While there are published papers on reflective writing by medical students and residents, none exist in the area of social pediatrics or address how social determinants of health (SDoH) impact an individual's overall health. The research question for this study is: During the Social Pediatrics rotation, how has exploring SDoH led to changes in residents' awareness of their own practice?Methods: Grounded theory was used as a methodology to analyze 35 reflections from the residents who had submitted them as an assignment to their preceptor in the Social Pediatrics rotation. In addition, 10 semi-structured telephone interviews were conducted to further understand residents' perceptions. Interviews were transcribed verbatim and analyzed using thematic analysis.Results: To analyze these reflections and our interviews, our analysis was guided by grounded theory using open, axial, and selective coding, and revealed the following themes: (1) bias, (2) emotional response to experiences, (3) systemic challenges, (4) community, (5) frustration/hopelessness, (6) “everyone is doing their best,” and (7) advocacy. Interview data reinforced themes of bias, and systemic challenges and advocacy were also apparent in the interviews, in addition to the following themes: (1) exposures to new populations and locations, (2) increased knowledge of specific populations and resources, and (3) impact of SDoH on overall health.Conclusions: Themes that emerged from residents' experiences during their Social Pediatrics rotation highlight the importance of enhancing residents' education regarding SDoH. Analysis of residents' written reflection assignments and follow-up interviews highlighted the importance of fostering learning experiences not typically encountered in traditional clinical learning environments and the value of reflective practice in physician development.K. Connors1, M. Rashid1, J. Walton1, M. Chan2, B. Islam11University of Alberta, Edmonton, AB, Canada; 2University of British Columbia, Vancouver, BC, Canada