Competency-based Medical Education Research Articles

"Learner’s Perspectives On Competency-Based Medical Education In Community Medicine: A Study At Noida International Institute Of Medical Sciences (NIIMS) In District Gautam Budh Nagar"

"Learner’s Perspectives On Competency-Based Medical Education In Community Medicine: A Study At Noida International Institute Of Medical Sciences (NIIMS) In District Gautam Budh Nagar"

Competency Based Medical Education (CBME) in India: Clinicians’ Multifaceted Challenges

Competency Based Medical Education (CBME) in India: Clinicians’ Multifaceted Challenges

Use the right words: evaluating the effect of word choice and word count on quality of narrative feedback in ophthalmology competency-based medical education assessments

Background: The purpose of this study was to investigate the effect of word choice on the quality of narrative feedback in ophthalmology resident trainee assessments following the introduction of competency-based medical education at Queen’s University. Methods: Assessment data from July 2017-December 2020 were retrieved from ElentraTM (Integrated Teaching and Learning Platform) and anonymized. Written feedback was assigned a Quality of Assessment for Learning (QuAL) score out of five based on this previously validated rubric. The correlation between QuAL score and specific coaching words was determined using a Spearman’s Rho analysis. Independent samples t-tests were used to compare the QuAL score when a specific word was used, and when it was absent. Results: A total of 1997 individual assessments were used in this analysis. The number of times the identified coaching words were used within a comment was significantly and positively associated with the total QuAL score, with the exception of “next time” (rho=0.039, p=0.082), “read” (rho = 0.036, p = 0.112), “read more” (rho = -0.025, p = 0.256) and “review” (rho = -0.017, p = 0.440). The strongest correlations were for “continue” (rho = 0.182, p < 0.001), “try(ing)” (rho = 0.113, p < 0.001) and “next step” (rho = 0.103, p < 0.001). The mean value of the QuAL score increased when coaching words were used vs. not used with the largest mean difference of 1.44 (p < 0.001) for “reflect”. A clear positive relationship was demonstrated between word count and QuAL score (rho = .556, p < 0.001). Conclusions: The use of certain coaching words in written comments may improve the quality of feedback.

Evaluating the Quality of Narrative Feedback for Entrustable Professional Activities in a Surgery Residency Program.

We assessed the quality of narrative feedback given to surgical residents during the first five years of Competency-Based Medical Education (CBME) implementation. CBME requires ongoing formative assessments and feedback on learners' performance. We conducted a retrospective cross-sectional study using assessments of Entrustable Professional Activities (EPAs) in the Surgical Foundations curriculum at Queen's University from 2017-2022. Two raters independently evaluated quality of narrative feedback using the Quality of Assessment of Learning (QuAL) Score (0-5). A total of 3,900 EPA assessments were completed over 5 years. Fifty-seven percent (2229/3900) of assessments had narrative feedback documented with a mean QuAL score of 2.16±1.49. Of these, 1614 (72.4%) provided evidence about the resident's performance, 951 (42.7%) provided suggestions for improvement, and 499/2229 (22.4%) connected suggestions to the evidence. There was no meaningful change in narrative feedback quality over time (r=0.067, P=0.002). Variables associated with lower quality of narrative feedback include: Attending role (2.04±1.48) compared to medical student (3.13±1.12, P<0.001) and clinical fellow (2.47±1.54, P<0.001), concordant specialties between the assessor and learner (2.06±1.50 vs. 2.21±1.49, P=0.025), completion of the assessment one month or more after the encounter versus one week (1.85±1.48 vs. 2.23±1.49, P<0.001), and resident entrustment versus not entrusted to perform the assessed EPA (2.13±1.45 vs. 2.35±1.66; P=0.008). The quality of narrative feedback was similar for assessments completed under direct and indirect observation (2.18±1.47 vs. 2.06±1.54; P=0.153). Just over half of the EPA assessments of surgery residents contained narrative feedback with overall fair quality. There was no meaningful change in the quality of feedback over 5 years. These findings prompt future research and faculty development.

Perspectives of medical faculty regarding implementation of Competency-Based Medical Education (CBME) in Uttarakhand, India

: Competency Based Medical Education (CBME) introduced in 2019, by the National Medical Commission Erstwhile Medical Council of India, involves restructuring the medical training and planning of curriculum and assessment with a focus on developing competent Indian Medical Graduates.: To assess the perception of medical faculties towards the implementation of new Competency-Based Medical Education.: A cross-sectional study was conducted for 1 Month (July 2023). 42 Faculty Members were enrolled in the study after obtaining their informed consent. A pretested semi-structured questionnaire using Google Forms was used to assess the perception of faculty regarding CBME implementation. Data was entered using MS Excel. Statistical analysis was performed using Stata software (version 11.0, Stata, College Station, Texas). Out of 42 Faculty members,16 were Professors and 26 were Associate Professor. The majority (97.6%) of faculty members were trained in medical education training through various workshops. The majority (69%) of faculty members agreed that CBME is essential as per changing time scenario. Although 81% of faculty members were not happy with frequent changes in regulations as released by NMC after the implementation of CBME. The majority (76%) of faculty members also shared that CBME changed their way of Teaching. Faculty members also unanimously shared that faculty and infrastructure recommended by NMC are less and there is a need to revise recommended requirements for effective implementation of CBME. Faculty members unanimously agreed that Competency-Based Medical Education is very much-needed in India as per evolving and challenging healthcare needs across the globe. However, there is a need to revise (increase) recommended manpower, and infrastructure by NMC for the effective implementation of CBME.

Assessment in Undergraduate Competency-Based Medical Education: A Systematic Review.

Studies that have methodically compiled the body of research on the competency-based medical education (CBME) assessment procedure and pinpointed knowledge gaps about the structure of the assessment process are few. Thus, the goals of the study were to create a model assessment framework for competency-based medical education that would be applicable in the Indian setting as well as to thoroughly examine the competency-based medical education assessment framework. PubMed, MEDLINE (Ovid), EMBASE (Ovid), Scopus, Web of Science, and Google Scholar were the databases that were searched. The search parameters were restricted to English language publications about competency-based education and assessment methods, which were published between January 2006 and December 2020. A descriptive overview of the included research (in tabular form) served as the foundation for the data synthesis. Databases provided 732 records; out of which 36 fulfilled the inclusion and exclusion criteria. Thirty-six studies comprised a mix of randomized controlled trials, focus group interviews, and questionnaire studies, including cross-sectional studies, qualitative studies (03), mixed-method studies, etc. The papers were published in 10 different journals. The greatest number was published in BMC Medical Education (18). The average quality score for included studies was 62.53% (range: 35.71-83.33%). Most authors are from the UK (07), followed by the USA (05). The included studies were grouped into seven categories based on their dominant focus: moving away from a behavioristic approach to a constructive approach of assessment (01 studies), formative assessment (FA) and feedback (10 studies), the hurdles in the implementation of feedback (04 studies), utilization of computer or online based formative test with automated feedback (05 studies), video feedback (02 studies), e-learning platforms for formative assessment (04 studies), studies related to workplace-based assessment (WBA)/mini-clinical evaluation exercise (mini-CEX)/direct observation of procedural skills (DOPS) (10 studies). Various constructivist techniques, such as concept maps, portfolios, and rubrics, can be used for assessments. Self-regulated learning, peer feedback, online formative assessment, an online computer-based formative test with automated feedback, the use of a computerized web-based objective structured clinical examination (OSCE) evaluation system, and the use of narrative feedback instead of numerical scores in mini-CEX are all ways to increase student involvement in the design and implementation of the formative assessment.

Rethinking Informed Consent as an EPA in Surgery.

Over the past decade, entrustable professional activities (EPAs) have become an important element in the competency-based medical education movement. In this Commentary, the authors explore informed consent as an EPA within resident surgical training. In doing so, they foreground the concept of culture and reexamine the nature of trust and entrustment decisions from within a cultural framework. The authors identify role modeling and professional identity formation as core elements in the training process and suggest that faculty are sometimes better off using these tools than uncritically adopting a formal EPA framework for what is, in essence, a professionally oriented and values-based moral enterprise. They conclude that EPAs work best when they are developed at a local level, stressing the unique specialty and program culture as well as the care that must be taken when attempting to transfer notions of entrustment from the undergraduate medical education level to graduate medical education settings.

Transforming medical education in the 21st century: the role of competency-based medical education

Transforming medical education in the 21st century: the role of competency-based medical education

IJCM_395A: Exploring medical student’s perspective on Competency based medical education (CBME) curriculum at Government Medical College in Maharashtra.

Background: Competency based medical education (CBME) is implemented recently in India. The study was conducted to evaluate the perspectives of undergraduate medical students towards CBME curriculum at a Government Medical College in Maharashtra. CBME is helpful in expanding the dimensions besides basic medical knowledge to the core competencies of a high-grade skilled clinical graduate. Moreover, value addition with other competencies such as empathy, professionalism, altruism, communication skills, ethics, and humanities that focuses on health systems as a National goal ‘Health for All’. Objective: To explore medical student’s perspective on Competency based medical education (CBME) curriculum at Government Medical College in Maharashtra. Methodology: This cross-sectional descriptive study was conducted among undergraduate medical students at Dr. SCGMC, Nanded. A validated questionnaire was administered through Google link among medical students. Results: A total of 225 students has participated. Around 130 (58%) students are from first year and 96(43.2%) students opined that learning AETCOM skills are necessary and around 125(55.8%) students opined that there should be sometime for sports activities during week days also and 107(48.2%) students stated that early clinical exposure in the first year is beneficial. Conclusion: The CBME curriculum focuses on learning the competencies needed in clinical practice and provides standards and framework for measuring performance. The thrust of the new curriculum attempts to make medical education in India more outcomes oriented.

Motivations for Entrustable Professional Activity Assessment: Gaps Between Curriculum Theory and Resident Reality

Background Previous research demonstrates mixed reactions from residents toward competency-based medical education (CBME), and entrustable professional activities (EPAs) specifically. However, understanding what motivates residents to obtain EPAs may be vital to the longevity of CBME, given the emphasis on assessment for learning under this paradigm. Objective This study explored resident perspectives across 3 domains: motivation for obtaining EPAs, perceived importance of EPAs, and overall thoughts on CBME curriculum. Methods This was a sequential exploratory mixed-methods study involving 2 phases of data collection. Phase 1 was semi-structured interviews with residents enrolled in CBME at one Canadian institution from November 2019 to July 2020. Analyses included thematic and manifest content analysis. Phase 2 was an electronic close-ended survey to capture residents’ primary motivation for requesting EPAs and importance of EPAs for learning. Survey data were analyzed descriptively. Results Of 120 eligible residents, 25 (21%) and 107 (89%) participated in the interview and survey, respectively. Program requirement was the dominant motivation for obtaining EPAs. There was variability in perceived importance of EPAs on learning. Increased resident workload, gaming the system to maximize EPA scores, and lack of shared ownership from preceptors were cited as critiques of the curriculum. Survey responses corroborated interview findings. Conclusions Although many residents recognize the value of EPAs, the majority are not intrinsically motivated to seek out assessment under the current CBME framework.

IJCM_378A: Introduction of Elective modules Block 2 in undergraduate new Competency based Medical Education (CBME) Curriculum: Perception of students

Background: The new competency based medical education curriculum (CBME) introduced 2 months electives (2 blocks of 4 weeks) in the curriculum for undergraduates medical students during their study periods (at the completion of phase 3 part 1). Elective is an optional course, generally of short duration, where students can opt ‘electives’ as per their topic of interest, career preference and availability at institutes. During electives students get an opportunity to differentiate during medical training, experience applicative use of knowledge and develop applied skills under the supervision and guidance of preceptors. Objective: To assess the Students perception regarding Elective modules (Block-2) necessity and benefit. Methodology: This was a cross sectional study conducted in Department of Community Medicine, Pt JNM Medical college, Raipur and Associated tertiary care hospital Dr. BRAM hospital. Raipur, Chhattisgarh in which 181 undergraduates medical students were interviewed through online mode using google form and was circulated to all via WhatsApp group. Results: Total 181 study subjects were participated in which 56.4% male and 43.6% female. Sixty-eight percent study subjects agreed that elective posting increases their knowledge and skill. Most of the students 66(36.5%) perceived elective posting very good Conclusion: This study concluded that students had very good overall experience of elective posting which proved to be helpful in improving their knowledge, clinical skills and enhanced their learning opportunities.

IJCM_88A: Perception of the medical college faculty on new competency-based medical education curriculum for MBBS students

Background: Medical education needs to be adjusted to meet the fast- changing expectations of society and the resulting demands on daily medical practice. The Medical Council of India (MCI) has updated the Medical Curriculum based on CBME. Objectives: 1. To assess the perceptions of the faculty members of the Medical college on the competency-based medical education curriculum for the MBBS course 2. To assess the readiness to incorporate competency-based medical education in the medical education curriculum among faculty members Methodology: This cross-sectional study was set in Kasturba Medical College, Mangalore, among teaching staff of pre-clinical, para-clinical, and clinical departments. A total of 93 (pre-clinical-29, para-clinical-31, clinical=33) faculty members were requested to fill out the questionnaire. Results: Our study highlights inter-department coordination, time demands, and lack of faculty training as the major barriers to the implementation of CBME. Among all participants around 60% were females. Most of the participants had teaching experiences between 0-5 years. Almost 95% of the participants were aware of the new MCI 2019 curriculum. The majority of the participants rated their knowledge about CMBE as moderate level (38.7%). 61.3% of the faculty members consider themselves to be ready to incorporate CBМЕ. According to most faculty members, a moderate amount (50%) of training is required and sessions and workshops (68.8%) are the most preferred type of training. Conclusion: Most of the faculty members found that the CBME pattern will be effective in imparting skills to undergraduates. Most of the participants identified inter-department coordination, demands of time, and lack of training in evaluation and feedback to be the major barriers to implementing the CBME pattern. Most of the faculty members consider themselves to be ready for this change even though a lot of them find the implementation of CBME to be a difficult task.

IJCM_394A: Participation of Undergraduate Medical Students in Health Research and Factors Influencing It in a Medical College of Puducherry- A Cross-Sectional Study

Background: Developing, maintaining, and sustaining undergraduate medical students’ research involvement will enable future healthcare practitioners to practice more appropriate evidence-based clinical medicine. Objective: To assess the participation in health research and factors promoting it among undergraduate medical students in Puducherry. Methodology: A descriptive cross-sectional study was conducted among undergraduate medical students in Puducherry over a period of three months (August-October 2023). Systematic random sampling was applied to recruit 360 study participants based on Chellaiyan et al. A pre-tested, semi-structured questionnaire was used to collect the data, including socio-demographic characteristics, academic performance, participants’ attitudes, perceived motivational and barrier factors towards health research, and current involvement in health research. Data were entered in MS Excel 2019 and analyzed using SPSS v16.0. Results: A total of 360 participants were interviewed, of whom 51.6% were female and the majority (86.7%) were hostelers. Among the 360 study participants, only around one-tenth of the participants”41 (11.39%)” have participated in the health research. The factors of gender, competency-based medical education approach, and grade of the participants in the recent university exam were associated with participation in health research. The majority (90.4%) of the participants were highly motivated to participate in health research. The main barriers identified are lack of funds, collaboration, mentorship, time to conduct, and systematic training in the research methodology. Conclusion: Participation in health research has doubled since the introduction of competency-based medical education. Still, participation could be improved by addressing the identified barriers. Studentship programs, young researcher awards, systematic training programs integrated into the curriculum, etc. could also compute the participation in health research among undergraduate medical students.

143 Training &amp; Sustaining: Training and learning collaborative outcomes across a statewide network for early diagnosis of children with autism

OBJECTIVES/GOALS: Community-based primary care autism diagnostic models are one promising solution to delays in autism diagnosis. Our objective is to describe the development and report on outcomes related to primary care professional (PCP) training and sustained engagement in a longitudinal learning collaborative across a statewide network for autism diagnosis. METHODS/STUDY POPULATION: We developed ADAPT (i.e., Accelerating the Diagnosis of Autism with Primary care Training), a training program to prepare PCPs to develop independent competency in evaluation of autism in children ages 14-48 months. ADAPT includes didactic and case-based modules and expert practice-based coaching delivered by a diagnostic specialist; following training PCPs participate in a longitudinal learning collaborative. Aligned with competency-based medical education standards, measures of autism evaluation knowledge and diagnostic competency are collected. RESULTS/ANTICIPATED RESULTS: To date, 13 PCPs have completed ADAPT didactic and practicum training reaching competency in independent autism evaluation. Clinicians demonstrated significant improvement in total autism knowledge following didactic training (p=.02). There was an overall trend toward increased scoring agreement on an autism observational assessment over case observations and practicum evaluations. Similarly, PCPs demonstrated improved evaluation competence, moving on average from Advanced Beginner to Competent Performer as rated by expert trainers. Following training, PCPs attended 57% of monthly learning collaborative sessions. DISCUSSION/SIGNIFICANCE: Training PCPs to deliver autism evaluations as part of community-based models of care is a promising solution to address access and waitlist challenges. ADAPT is an intensive, standard PCP training model which results in achievement of independent competency and sustained engagement in in autism evaluation.

In EPAs we trust, is quality and safety a must? A cross-specialty analysis of entrustable professional activity guides

Purpose The inclusion of quality improvement (QI) and patient safety (PS) into CanMEDS reflects an expectation that graduating physicians are competent in these areas upon training completion. To ensure that Canadian postgraduate specialty training achieves this, the translation of QI/PS competencies into training standards as part of the implementation of competency-based medical education requires special attention. Methods We conducted a cross-specialty, multi-method analysis to examine how QI/PS was incorporated into the EPA Guides across 11 postgraduate specialties in Canada. Results We identify cross-specialty variability in how QI/PS is incorporated, positioned, and emphasized in EPAs and milestones. QI/PS was primarily referenced alongside clinical activities rather than as a sole competency or discrete activity. Patterns were characterized in how QI/PS became incorporated into milestones through repetition and customization. QI/PS was also decoupled, conceptualized, and emphasized differently across specialties. Conclusions Variability in the inclusion of QI/PS in EPAs and milestones has important implications considering the visibility and influence of EPA Guides in practice. As specialties revisit and revise EPA Guides, there is a need to balance the standardization of foundational QI/PS concepts to foster shared understanding while simultaneously ensuring context-sensitive applications across specialties. Beyond QI/PS, this study illuminates the challenges and opportunities that lie in bridging theoretical frameworks with practical implementation in medical education, prompting broader consideration of how intrinsic roles and emergent areas are effectively incorporated into competency-based medical education.

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Navigating competency-based medical education in ophthalmology: Addressing challenges and charting future trajectories

Based Medical Education (CBME) has emerged as a transformative approach to medical training, emphasizing skills acquisition and holistic competence over traditional knowledge retention. Ophthalmology, as a specialized field, presents unique challenges and opportunities within this framework. This article critically examines the implementation of CBME in ophthalmology, highlighting challenges faced and proposing strategies for enhancement. By addressing stakeholder perspectives and strategic interventions, this viewpoint aims to foster a comprehensive understanding of CBME's implications for ophthalmological education.

Dual purposes by design: exploring alignment between residents' and academic advisors' documents in a longitudinal program.

Longitudinal academic advising (AA) and coaching programs are increasingly implemented in competency based medical education (CBME) to help residents reflect and act on the voluminous assessment data they receive. Documents created by residents for purposes of reflection are often used for a second, summative purpose-to help competence committees make decisions-which may be problematic. Using inductive, thematic analysis we analyzed written comments generated by 21 resident-AA dyads in one large internal medicine program who met over a 2year period to determine what residents write when asked to reflect, how this aligns with what the AAs report, and what changes occur over time (total 109 resident self-reflections and 105 AA reports). Residents commented more on their developing autonomy, progress and improvement than AAs, who commented far more on performance measures. Over time, residents' writing shifted away from intrinsic roles, patient care and improvement towards what AAs focused on, including getting EPAs (entrustable professional activities), studying and exams. For EPAs, the emphasis was on getting sufficient numbers rather than reflecting on what residents were learning. Our findings challenge the practice of dual-purposing documents, by questioning the blurring of formative and summative intent, the structure of forms and their multiple conflicting purposes, and assumptions about the advising relationship over time. Our study suggests a need to re-evaluate how reflective documents are used in CBME programs. Further research should explore whether and how documentation can best be used to support resident growth and development.

Navigating the Roadmap for Trauma-Informed Medical Education: Application of Undergraduate Medical Education Competencies.

Trauma is common in the United States, increases risk of long-term adverse health effects, and individuals who experience it often find seeking medical care difficult. Trauma-informed care (TIC) builds trust and fosters healing relationships between clinicians and patients; however medical education has lacked consistent training in TIC. Using recently published competencies for undergraduate medical education (UME), this manuscript provides curricular examples across 8 domains to assist faculty in developing educational content. The authors identified published curricula for each of the 8 competency domains using a published search strategy and publicly available database. Inclusion criteria were published works focused on UME in the United States; abstracts and curricula not focused on UME were excluded. The authors used a consensus-based process to review 15 eligible curricula for mapping with the competencies. Of 15 published UME curricula, 11 met criteria and exemplify each of the 8 UME competency domains. Most of the available curricula fall into the Knowledge for Practice and Patient Care domains. Most were offered in the first 2 years of medical school. Competency-based medical education for TIC is new, and most current educational offerings are foundational in nature. Additional innovation is needed in the competency domains of Professionalism, Systems-Based Practice, Interprofessional Collaboration, and Personal/Professional Development. This manuscript offers a set of curricular examples that can be used to aid efforts at implementing TIC competencies in UME; future work must focus on improving assessment methods and developmental sequencing as more students are exposed to TIC principles.

Clinician Educator Milestones: Assessing and Improving Educators' Skills.

The importance of the clinician educator (CE) role in the delivery of competency-based medical education is well-recognized. There is, however, no formal mechanism to identify when faculty have the knowledge, skills, and attitudes to be successful CEs. In 2020, the Accreditation Council for Graduate Medical Education, Accreditation Council for Continuing Medical Education, Association of American Medical Colleges, and American Association of Colleges of Osteopathic Medicine convened a workgroup of 18 individuals representing multiple medical specialties and diverse institutions in the United States, including nonphysician educators, a medical student, and a resident, to develop a set of competencies, subcompetencies, and milestones for CEs.A 5-step process was used to create the Clinician Educator Milestones (CEMs). In step 1, the workgroup developed an initial CEM draft. Through brainstorming, 141 potential education-related CE tasks were identified. Descriptive statements for each competency and developmental trajectories for each subcompetency were developed and confirmed by consensus. The workgroup then created a supplemental guide, assessment tools, and additional resources. In step 2, a diverse group of CEs were surveyed in 2021 and provided feedback on the CEMs. In step 3, this feedback was used by the workgroup to refine the CEMs. In step 4, the second draft of the CEMs was submitted for public comment and the CEMs were finalized. In step 5, the final CEMs were released for public use in 2022.The CEMs consist of 1 foundational domain that focuses on commitment to lifelong learning, 4 additional domains of competence for the CE in the learning environment, and 20 subcompetencies. These milestones have many potential uses for CEs, including self-assessment, constructing learning and improvement plans, and designing systematic faculty development efforts. The CEMs will continue to evolve as they are applied in practice and as the role of CEs continues to grow and develop.