ACGME Competencies Research Articles

Exploring Clinical Competencies and Ethical Reflections: A Multi-Patient Simulation-Based Training Program for Emergency Medicine Residents

Objectives: This study aims to assess the proficiencies, ethical considerations, and crisis management abilities of emergency medicine residents through a multi-patient simulation program. The research investigates the impact of this training on developing skilled emergency care providers. Methods: Fifteen emergency medicine residents at various training levels were evaluated by two specialists across five simulation scenarios aligned with ACGME competencies. Performance in clinical decision-making, communication, and teamwork was assessed using a 5-point Likert scale and Google Forms, following obtaining informed consent. Results: The study lasted three days and included briefings, simulations, and debriefings. Residents demonstrated differing performance levels, excelling in patient history taking and data reassessment but facing challenges in forensic inquiries, ethical dilemmas, and team communication under pressure. Residency experience showed positive correlations with certain performance aspects. Conclusions: Variations in residents' performances highlight the complexities of emergency medicine and the role of simulation in identifying educational gaps. The debriefing session emphasized the importance of ethical practice and effective team communication. The debriefing session highlighted ethical conduct and effective team communication, advocating simulation-based training to improve emergency medicine competencies. Additionally, the session gathered information that would guide future research endeavors and the development of educational policies.

Differences in Resident Self-Evaluation and Clinical Competency Committee Evaluation Using ACGME Milestone Versions 1.0 and 2.0

Intentionally self-driven professional development of surgical resident physicians is a hallmark of surgical training and is expected to gain further traction as Entrustable Professional Activities (EPAs) become the new paradigm for surgical education. We aimed to analyze how surgical residents rate themselves as compared to the evaluation of the Clinical Competency Committee using ACGME Milestones Version 1 (M1.0) and Version 2 (M2.0). We asked 22 general surgical trainees for self-evaluation of Milestones (both M1.0 and M2.0) from 2017 semiannually to 2022. ACGME-required Milestone evaluations by the Clinical Competency Committee (CCC) were independently performed after the time window for resident self-evaluation. Neither trainees nor CCC were aware of the other party's evaluations. There were 1552 paired data available for evaluating individual competencies by both trainees and CCC. Paired Wilcoxon signed-rank tests were then performed among the corresponding pairs. MercyOne Des Moines Medical Center, Des Moines, IA; Teaching tertiary referral center. Twenty-two general surgical trainees at this hospital and 28 faculty surgeons participated in this study. The average self-evaluation of surgical residents was lower in the M1.0 cohort compared to the corresponding CCC evaluation (1.96 ± 0.72 vs. 2.11 ± 0.67; p < 0.001). M1.0 self-assessments and CCC-assessments were statistically similar for ICS (p = 0.548) and PROF (p = 0.554) competencies and differed for MK (p < 0.001), PBLI (p < 0.001), PC (p < 0.001), SBP (p = 0.008). On the contrary, the M2.0 cohort demonstrated higher average self-evaluation of surgical residents compared to the corresponding CCC evaluation (2.75 ± 0.87 vs. 2.12 ± 0.97; p < 0.001). Significant differences were observed for all 6 ACGME competencies using M2.0 self-assessments and CCC-assessments (all p < 0.001). Multivariate regression modeling (p < 0.001, R2 = 0.255) predicted the degree of discordance between self-assessment and CCC-assessed achievement of competencies with a significant effect of gender (baseline male: coef = -0.232, p < 0.001), PGY level (-0.083 per year, p < 0.001) and Milestone version (0.831, p < 0.001). A significant interaction exists for all gender/Milestone combinations except for the female trainees with M1.0. The difference between self-evaluated Milestone achievement and faculty-driven CCC evaluation of surgical resident physician performance is more evident in Milestones 2.0 than in Milestones 1.0. Residents self-evaluate higher compared to faculty using Milestones 2.0. This discrepancy is seen among both genders and is more pronounced among male residents overestimating core competencies with M2.0 self-evaluation than formal CCC assessment.

Expanding scope of Kirkpatrick model from training effectiveness review to evidence-informed prioritization management for cricothyroidotomy simulation

Modified Kirkpatrick model has been adopted to evaluate training effectiveness by 6 categories, including activity accounting (training objectives/success in organization change) at Level-0, reaction (satisfaction) at Level-1, learning (acquisition of surgical airway skills) at Level-2, behavior (post-training change in personal strengths) at Level-3, result (organizational or clinical outcomes) at Level-4, and Return on Investment (ROI) or Expectation (ROE) (monetary and societal values following training and other quality and safety related measures) at Level-5. The purpose of this hospital-based prospective observational study was twofold: i) To evaluate potential impacts on monetary and societal values and successful organization change following implementation of advanced Cricothyroidotomy simulator and standardized curriculum in healthcare simulation training, ii) To inform decisions of resource allocation by reviewing overall values and prioritization strategies for i) general surgeon/emergency physician ii) with seniority >5 years and iii) prior porcine training experience based on findings at Kirkpatrick Level-0, Level-4, and Level-5. Seventy doctors and 10 nurses completed Cricothyroidotomy training and follow-up questionnaires within 2021/22. All training usability scoring measured by Scales of Emergency Surgical Airway Simulator (SESAS-17) achieved over 4 out of 5 (Level-4) with effects in favor of emergency physicians or general surgeons (p < .5), regardless of seniority and prior training experience. Success in organization change (Level-0) and cost-effectiveness (Level-5) were hypothetically established using theoretical framework of Gleicher's formula and Roger's Diffusion of Innovation Theory. Overall training effectiveness, in terms of advantage in usability, cost-benefits and successful organizational changes, provided sound evidence to support continuous investment of new curriculum and innovative simulator and “Surgeon-and-emergency-physician-first” policy when it comes to resources allocation strategies for Cricothyroidotomy training. [ACGME competencies: Practice Based Learning and Improvement, Systems Based Practice.]

Quality of life among otolaryngology and head and neck surgery residents in Saudi Arabia: A cross-sectional study

Introduction: The surgical residency training programs have the nature of long work hours and a stressful environment that might lead to disturbances in the quality of life of the residents. Surgical residents’ performance might be affected by the poor quality of life. This study aimed to assess the quality of life of otolaryngology and head and neck surgery residents, in Saudi Arabia. Subjects and methods This is a cross-sectional study conducted among otolaryngology and head and neck surgery residents in Saudi Arabia. A self-administered questionnaire was distributed among the targeted residents. The questionnaire is primarily composed of the socio-demographic data and the work-related quality of life (WRQoL) questionnaire. The WRQoL was composed of 6 domains as General Well-Being (GWB), Home-Work Interface (HWI), Job and Career Satisfaction (JCS), Control at Work (CAW), Working Conditions Satisfaction (WCS ), and Stress at Work (SAW). Results: The overall work-related quality of life was good among 43.6% of the residents. Among its domains, WCS showed better results with a 55.3% high level while SAW was the least (25.5%). Older age group (30 years) were observed to have a significantly higher mean score in JCS (p=0.040), CAW (p=0.002) and WRQoL (p=0.027). Smoking residents showed a higher mean score in JCS (p=0.023) and HWI (p=0.001). It is interesting to note that increasing residency levels were more associated with increasing CAW scores (p=0.003). Conclusion: The work-related quality of life among otolaryngology, head and neck surgical residents was adequate. Older residents demonstrated a better quality of life than younger residents. ACGME competencies: Medical knowledge, Patient care, Interpersonal skills communication. Keywords: Quality of life, WRQoL, otolaryngology, head and neck surgery, residents

How Do We Choose? A Review of Residency Application Scoring Systems

As the USMLE Step 1 Board exam moves to a pass/fail system there will be fewer objective measurements available to evaluate students applying to residency programs. Thus, there is a need for a reliable, validated method of screening applicants based on all areas of their application. To this end, we conducted a literature review search to examine previously described residency application screening tools. A PubMed search was conducted using the keywords "residency," "applicant," "scoring," "algorithm," and "ranking." The search was limited to the last 10 years, and only articles written in English with the full-text available were included. The initial search yielded 512 results. Titles and abstracts were evaluated for inclusion and 11 articles met criteria for full-text evaluation. An additional 6 articles were excluded with reason following the full-text evaluation. A total of 5 papers were included in our descriptive analysis. Villwock et al. used the open-source STAR algorithm to create an initial interview list based on program-specific desirable attributes. Bowe et al. attempted to develop a screening tool based on the 6 ACGME competencies that could accurately predict a resident's performance. Similarly, Lyons et al. worked with an outside consulting firm to develop a screening tool that used several situational judgment questions to assess desired competencies for first year residents. Schenker et al. developed an evaluation process that used a combination of a standardized screening tool and semistructured interviews to produce a final rank list. Hu et al. created a screening tool for pharmacology residency applicants based on specific domains. Several residency application scoring systems have been evaluated for use in the initial screening process, but there is no consensus on which system is superior and whether or not those systems are successful in selecting the "best" candidates.

Clinical Case-Conference Blogs: Integrating Clinical Librarians to Enhance Resident Education and Enforce ACGME Competencies.

Resident conferences are primary educational endeavors for trainees and faculty alike. We describe the development of collaborative clinician-librarian educational blogs within the Internal Medicine (2009), Pediatrics (2012), and General Surgery (2018) residency programs. Clinical librarians attended resident conferences and generated evidence-based blog posts based on learning topics and clinical questions encountered during the conferences. In the decade since introduction of the blogs, this partnership has resulted in over 2000 blog posts and generated over 1800 individual views per month. The development of a clinical librarian-managed blog serves as a relevant resource for promoting evidence-based practices within a case-based learning curriculum, engages interdisciplinary collaboration through existing resources, and is generalizable across various clinical practice disciplines and trainees.

Geriatric Fast Facts: Four Years of Growth and Reach

Clinical teachers are increasingly challenged to find the time to provide point of care education for learners. To meet this challenge, an interprofessional team from competing health care systems created and sustained Geriatric Fast Facts (GFF): easily accessible, concise 1-2 page topic summaries for clinical teachers to use (in lieu of the mini-lecture) with learners at the point of care. Designed by geriatrics educators in consultation with IT experts, we launched a mobile enabled website that is indexed and searchable by free text or topic, organ system, ACGME competency, disease, and the “underlying science” for the disease/illness. GFF topics are authored by subject matter experts with peer review by senior geriatricians. Brief quizzes test learners’ knowledge with score reports. Our 4-year results reveal that: 1) “Geriatric Fast Facts” appear in the top Google 10 listing; 2) Search engines account for 39% of site traffic; 3) 60% of unique users (N=29,000) are via direct link; 4) 19% via referral from another site. Our “bounce rate” (55%) is ideal as users quickly gain the information sought - affirmed by our session duration which averages < 2 minutes. Our Google Analytics results reflect steady growth and international reach: 73% of sessions are from the U.S., 5% from Canada, 22% international and growing! In summary with our population continuing to age and time for teaching being limited, GFFs are quick, accessible, evidence-based resources for point of care teaching.

Northwestern University Feinberg School of Medicine.

Northwestern University Feinberg School of Medicine.

25. WHAT'S THE EVIDENCE? EVALUATING THE IMPACT OF A RESIDENT EVIDENCE-BASED MEDICINE CURRICULUM

Background The ability to incorporate evidence-based medicine (EBM) into clinical practice is an ACGME competency, yet many pediatric residents have limited knowledge, comfort, and skill in this area. In addition, there is a lack of data regarding how residents apply EBM curricula to clinical practice. Objective: Describe the effect of an EBM curriculum on resident attitudes and clinical use of EBM. Methods We implemented a twice-weekly EBM curriculum to review key literature and guidelines. In this IRB-exempt mixed methods study, we surveyed residents, fellows, and faculty about use of EBM at baseline and 6 months later. Results were compared using Mann-Whitney U tests. We surveyed faculty twice weekly about residents’ EBM use on rounds. Residents participated in focus groups, which were audio-recorded, transcribed and coded using conventional content analysis to develop themes. Results Residents (N=61 pre- and 41 post-curriculum) reported increased: satisfaction with conferences (p=0.0037), importance of EBM (p = 0.0041), comfort generating a search question (p= 0.0001), and citation of evidence on rounds (p Conclusion As a result of this EBM curriculum, trainees described increased use of EBM in clinical practice and better understanding of research methodology. Additionally, resident use of EBM reinforced plans, taught faculty, and at times, changed management. Ongoing work is needed to create a culture that promotes EBM.

Loma Linda University School of Medicine.

Loma Linda University School of Medicine.

Competencies and Feedback on Internal Medicine Residents' End-of-Rotation Assessments Over Time: Qualitative and Quantitative Analyses.

To examine how qualitative narrative comments and quantitative ratings from end-of-rotation assessments change for a cohort of residents from entry to graduation, and explore associations between comments and ratings. The authors obtained end-of-rotation quantitative ratings and narrative comments for 1 cohort of internal medicine residents at the University of Illinois at Chicago College of Medicine from July 2013-June 2016. They inductively identified themes in comments, coded orientation (praising/critical) and relevance (specificity and actionability) of feedback, examined associations between codes and ratings, and evaluated changes in themes and ratings across years. Data comprised 1,869 assessments (828 comments) on 33 residents. Five themes aligned with ACGME competencies (interpersonal and communication skills, professionalism, medical knowledge, patient care, and systems-based practice), and 3 did not (personal attributes, summative judgment, and comparison to training level). Work ethic was the most frequent subtheme. Comments emphasized medical knowledge more in year 1 and focused more on autonomy, leadership, and teaching in later years. Most comments (714/828 [86%]) contained high praise, and 412/828 (50%) were very relevant. Average ratings correlated positively with orientation (β = 0.46, P < .001) and negatively with relevance (β = -0.09, P = .01). Ratings increased significantly with each training year (year 1, mean [standard deviation]: 5.31 [0.59]; year 2: 5.58 [0.47]; year 3: 5.86 [0.43]; P < .001). Narrative comments address resident attributes beyond the ACGME competencies and change as residents progress. Lower quantitative ratings are associated with more specific and actionable feedback.

The Initial 360-degree Evaluation Tool (360-1) Addressed 15 Specific Skills to Cover all Six ACGME Competencies

The Initial 360-degree Evaluation Tool (360-1) Addressed 15 Specific Skills to Cover all Six ACGME Competencies

A surgical residency preparatory course for senior medical students leads to earlier independence in ACGME competencies

BackgroundIt is unknown whether surgery residency preparatory courses lead to earlier independent practice. MethodsA four-week surgical residency preparatory course was offered to graduating medical students. Upon entering residency, participants reported supervised and unsupervised performance of patient management and procedural competencies. Those who participated in the course (Group A) were compared with graduates from our institution who did not participate but entered surgery residency (Group B) and with residents from other medical schools in the same program as Group A (Group C). Time to independence was observed. ResultsGroup A achieved independence earlier than Group B in 15/18 (83.3%), earlier than Group C in 14/18 (77.8%) and earlier than both in 12/16 (75%) competencies. Independence occurred 43.6 days earlier than Group B (range 6–112 days) and 49 days earlier than Group C (range 11.5–165 days). ConclusionA surgical residency preparatory course led to earlier independent performance of the ACGME recommended patient management and procedural competencies compared to students from our institution and others who did not participate in such a course.

Residents provide feedback to their clinical teachers: Reflection through dialogue

Background: Physicians play a crucial role in teaching residents in clinical practice. Feedback on their teaching performance to support this role needs to be provided in a carefully designed and constructive way.Aims: We investigated an evaluation system for evaluating supervisors and providing formative feedback.Method: In a design based research approach, the ‘Evaluation and Feedback For Effective Clinical Teaching System’ (EFFECT-S) was examined by conducting semi-structured interviews with residents and supervisors of five departments in five different hospitals about feedback conditions, acceptance and its effects. Interviews were analysed by three researchers, using qualitative research software (ATLAS-Ti).Results: Principles and characteristics of the design are supported by evaluating EFFECT-S. All steps of EFFECT-S appear necessary. A new step, team evaluation, was added. Supervisors perceived the feedback as instructive; residents felt capable of providing feedback. Creating safety and honesty require different actions for residents and supervisors. Outcomes include awareness of clinical teaching, residents learning feedback skills, reduced hierarchy and an improved learning climate.Conclusions: EFFECT-S appeared useful for evaluating supervisors. Key mechanism was creating a safe environment for residents to provide honest and constructive feedback. Residents learned providing feedback, being part of the CanMEDS and ACGME competencies of medical education programmes.

ACGME Competencies Should Be Required of Our Residencies, Not Just Our Residents

ACGME Competencies Should Be Required of Our Residencies, Not Just Our Residents

CanMEDS and other outcomes

As I indicated in a previous editorial (Norman 2007), the world of medical education is increasingly focused, for better or worse, on performance outcomes. One of the most influential taxonomies of outcomes is the CanMEDS roles, developed by the Royal College of Physicians and Surgeons of Canada beginning in 1993 (Frank et al. 1996). It is not alone, of course; we also have the ACGME competencies in the US, The Scottish Doctor and ‘‘Tomorrow’s Doctor’’ in the UK, and several others I don’t know about. But for reasons that I cannot begin to understand, the most widely adopted framework is the Canadian one, officially adopted by 16 countries. To set the stage, the CanMEDS competencies, eight in number: Medical Expert/Clinical Decision-maker, Communicator, Collaborator, Manager, Health Advocate, Scholar, Professional are usually portrayed as a flower with Medical Expert at the centre surrounded by the other 7 roles. One wonders if the dissemination success of the competencies relate in part to the powerful visual appeal of the flower. In this issue, we have two papers related to the CanMEDS roles, which, while using complementary methods, deliver a consistent message. The study by Whitehead et al. (2011) is quite unusual and a really interesting model for research. They recognized that the CanMEDS roles evolved from a large project involving all 5 (then) Ontario medical schools—the Educating Future Physicians of Ontario (EFPO) project, which in turn began in the late 1980’s as a response to a bitter physician strike when government legislated against ‘‘extra-billing’’, i.e. private patients. The ground was fertile for a redefinition of what patients, in Ontario, really wanted from their physicians. Whitehead et al. combed through the archives of the EFPO project, box after box, in the University of Toronto, using Foucauldian discourse analysis. Their findings reveal a disconnect between the overt goal—information to better inform medical schools about the expectations of patients, and the covert message of a profession under siege:

Can Moral Reasoning Predict General Surgery Residents' Clinical Competence?

When selecting residents for residency programs, there has been little success using standard academic criteria to predict their clinical performance. However, numerous studies in the past have found linkages between the nonstandard variable moral reasoning as measured by the instrument Defining Issues Test (Version 2; DIT-2) and clinical decision making with higher levels of moral reasoning linked to better clinical performance. This study sought to determine whether this important linkage exists in surgery residents. The DIT-2 was administered to 20 surgical residents to assess level of moral reasoning, and data were collected on all end-of-rotation evaluations of residents by faculty to measure resident clinical competence. Candidate rank list data were examined from the past 5 years to determine linkages between moral development as measured by the DIT-2 and clinical competence. Correlation analyses, univariate regression, and stepwise multivariate regression were carried out to assess the relationships of moral reasoning as measured by the DIT-2 with other study variables. Analysis of data indicates a low correlation between DIT-2 scores and clinical competence. Rank list order showed a slight correlation with resident DIT-2 scores. Rank position was weakly correlated with individual competencies and aggregated scores for all competencies as measured by faculty evaluations. Rank position coupled with DIT-2 scores were predictive of 4 of the 6 ACGME competencies and predictive of clinical competence as measured by aggregated scores for all competencies. No linkages emerged between DIT-2 scores and clinical competence. This reasons for this may include the size of the population studied and the assumption that end-of-rotation evaluations measure resident clinical competence. There were significant relationships demonstrated when DIT-2 scores were linked with resident ranking. Future research should be continued in this area but with more rigorous instrumentation to measure clinical performance and a larger sample size.

Emotional Intelligence and the ACGME Competencies.

Residency programs desire assessment tools for teaching and measuring resident attainment of the Accreditation Council for Graduate Medical Education competencies, including interpersonal and communication skills. We sought to evaluate the use of emotional intelligence (EI) assessment and training tools in assessing and enhancing interpersonal and communication skills. We used a quasi-experimental design, with an intervention and control group composed of 1 class each of family medicine residents. The intervention was EI coaching. The assessment used the Emotional and Social Competence Inventory, a 360-degree EI survey consisting of self and other (colleague) ratings for 12 EI competencies. There were 21 participants in each of the 3 assessments (test, posttest, and control). Our EI coaching intervention had very limited participation due to a lack of protected time for EI coaching and residents' competing obligations. Return rates for self surveys were 86% to 91% and 66% to 68% for others. On all 3 trials, ratings by others were significantly higher than self ratings for every competence (range, P < .001-.045). None of the self ratings by the intervention group increased significantly for any of the competencies. None of the intervention group self ratings increased significantly on posttesting, whereas ratings by others increased significantly for coach/mentor (P < .001). The teamwork rating decreased significantly on both self and other ratings (P < .001). Achievement orientation was the highest intervention group posttest rating, and teamwork was the lowest. EI is a necessary skill in today's health care environment, and our study found that a tool from another sector was useful in assessing resident EI skills. Because our EI coaching intervention was unsuccessful, the effects of coaching on interpersonal and communication skills could not be assessed.

P63 Morbidity and mortality conference using ACGME competencies: a case study in eclampsia

(1) Persistence of aberrant CVD risk factors and an accelerated progression of IMT. (2) Lower levels of circulating endothelial markers and impaired numbers of circulating endothelial progenitor cells (EPCs), indicating endothelial cell dysfunction and impaired vascular repair. Methods: 18 formerly preeclamptic women and 16 controls were included in our follow-up (study 2). Ultrasound measurements of carotid and femoral arteries and various classical cardiovascular risk factors were recorded. Blood samples were taken and whole blood was stained using anti-CD34, anti-CD133 and anti-vascular growth factor receptor antibodies. Flow cytometry was used to quantify EPCs (triple positive cells). Circulating endothelial markers (von Willebrand factor (vWF), ICAM-1) and inflammatory markers (hsCRP) were measured in plasma using ELISA. Results: See Table 1. Conclusion: This study suggests a transient adaptive response of the vasculature to a preeclamptic pregnancy which normalizes within years. Otherwise, the presence and persistence of cardiovascular risk factors in formerly preeclamptic women emphasizes the need to follow up these high-risk women in future.

Ponce School of Medicine

Ponce School of Medicine