Entrustable Professional Activities Research Articles

Entrustable Professional Activities in Palliative Medicine: A Faculty and Learner Development Activity.

Background: Faculty development (FD) is critical to the implementation of competency-based medical education (CBME) and yet evidence to guide the design of FD activities is limited. Our aim with this study was to describe and evaluate an FD activity as part of CBME implementation. Methods: Palliative medicine faculty were introduced to entrustable professional activities (EPAs) and gained experience estimating a learner's level of readiness for entrustment by directly observing a simulated encounter. The variation that was found among assessments was discussed in facilitated debrief sessions. Attitudes and confidence levels were measured 1 week and 6 months following debriefs. Results: Participants were able to use the EPA framework when estimating the learner's readiness level for entrustment. Significant improvements in attitudes and level of confidence for several knowledge, skill, and behavior domains were maintained over time. Conclusions: Simulated direct observation and facilitated debriefs contributed to preparing both faculty and learners for CBME and EPA implementation.

Identifying Community-Based Entrustable Professional Activities for Medical Students Through a Modified Delphi Process.

Recognition of the need for medical education to train physicians who are skilled at supporting population health and work beyond traditional health care settings is growing. Entrustable professional activities (EPAs) for medical students typically have centered around activities taking place in the clinical workplace; however, EPAs that involve working with community members in community contexts have not been clearly established. We used a three-stage online modified-Delphi method to identify community-based EPAs for University of Wisconsin School of Medicine and Public Health medical students. We recruited key stakeholders to participate and asked them, based on their experience, to generate a list of community-based tasks that they believed graduates should be trusted to perform. Subsequently, using a five-point anchored Likert scale (1=strongly disagree to 5=strongly agree), we asked participants to rate their level of agreement with each identified task becoming an EPA. An a priori definition of consensus was established. Twenty-two tasks reached consensus as potential community-based EPAs. The tasks with the highest mean ratings were "addressing trust issues with the medical community amongst the local population" (mean=4.71), "meeting with community members around a health topic" (mean=4.64), "identifying opportunities for disease prevention" (mean=4.64), and "identifying policies that impact community outcomes" (mean=4.57). The identified community-based tasks can support the augmentation of existing community-based curriculum and help identify areas for novel curriculum and assessment development. Lessons learned from this local effort could be helpful to other programs seeking to establish and refine community-based curricula.

Towards a flexible and personalised development of veterinarians and veterinary nurses working in a companion animal referral care setting.

In the Netherlands, the demand for veterinarians and veterinary nurses (VNs) working within referral care is rapidly growing and currently exceeds the amount of available board-certified specialists. Simultaneously, a transparent structure to guide training and development and to assess quality of non-specialist veterinarians and VNs working in a referral setting is lacking. In response, we developed learning pathways guided by an entrustable professional activity (EPA) framework and programmatic assessment to support personalised development and competence of veterinarians and VNs working in referral settings. Between 4 and 35 EPAs varying per discipline (n=11) were developed. To date, 20 trainees across five disciplines have been entrusted. Trainees from these learning pathways have proceeded to acquire new EPAs in addition to their already entrusted set of EPAs or progressed to specialist training during (n=3) or after successfully completing (n=1) the learning pathway. Due to their outcome-based approach, the learning pathways support flexible ways of development.

"Near Miss": A Mixed-Methods Analysis of Medical Student Assignments in Patient Safety.

The purpose of this study is to inform the curriculum for Entrustable Professional Activity 13 through analysis of fourth year medical student patient safety event assignments. From 2016 to 2021, students were asked to identify a patient safety event and indicate if the event required an incident report. Assignments were reviewed and coded based on Joint Commission incident definitions. Qualitative analysis was performed to evaluate incident report justification. There were 473 student assignments included in the analysis. Assignments reported incidents regarding communication, medical judgment, medication errors, and coordination of care. Students indicated only 18.0% (85/473) would warrant an incident report. Justification for not filing an incident report included lack of harm to the patient or that it was previously reported. Students were able to identify system issues but infrequently felt an incident report was required. Justifications for not filing an incident report suggest a need for a curriculum focused on the value of reporting near misses and hazardous conditions.

Setting the European standards for training in Physical and Rehabilitation Medicine.

The mission of the European Board of Physical and Rehabilitation Medicine (PRM) is to the ensure a consistent and high-level education for PRM physicians across Europe. An important action to accomplish this mission is the publication and continuous update of the European Training Requirements (ETRs) for the specialty of PRM. The first version of the ETRs for PRM was issued in 2017. The aim was to present the up-to-date training standards for trainees, trainers, and training institutions. The revision of the first ETRs, aiming to reflect the current standards in medical education and clinical advances in the field or PRM, started in 2022 and was completed in 2023. It was based on the Rehabilitation Competency Framework and the "Guide for using a contextualised competency framework to develop rehabilitation programmes and their curricula" published by the WHO in 2021. An important addition in the new version of the ETRs is the integration of Entrustable Professional Activities. In all endeavours of the creation of the ETRs, setting the highest standards of training in PRM was pursued.

Educational outcomes necessary to enter pharmacy residency training: 2023 update

AbstractThe 2023 Educational Affairs Committee reviewed relevant position statements and publications to update the 2014 ACCP position statement on the educational outcomes (EOs) necessary for pharmacy students to enter postgraduate year 1 (PGY1) residency training. The committee initially aligned entry‐level graduate outcomes with current PGY1 residency competencies achieved after postgraduate training. The committee then used these alignments to identify consistencies, insufficiencies, and gaps in preparation. This process revealed that the EOs and entrustable professional activities from curricular outcomes and entrustable professional activities and the North American Pharmacist Licensure Examination blueprint could serve as a partial list of outcomes necessary to enter postgraduate training; however, some of the outcomes necessary to enter residency were not consistently addressed. To improve residency readiness, the committee recommends that both academic programs and the individual learner evaluate and seek opportunities to address any curricular gaps in alignment. The findings also underscore the need for a single set of well‐defined and mutually agreed‐on educational competencies that evolve over time and embrace the knowledge, skills, and attitudes that demonstrate residency readiness.

Developing an electronic portfolio of learning for family medicine training in South Africa.

Workplace-based assessment has become increasingly crucial in the postgraduate training of specialists in South Africa, particularly for family physicians. The development of a Portfolio of Learning (PoL) has been a central focus within the discipline of family medicine for over a decade. Initially, a paper-based portfolio was adopted to collect evidence of learning for 50 out of 85 agreed exit-level outcomes. Stellenbosch University led the conversion of this portfolio into an electronic format, known as e-PoL, utilising Scorion software. The e-PoL was successfully implemented in the Western and Eastern Cape regions and was subsequently adopted nationally under the coordination of the South African Academy of Family Physicians. In 2023, the e-PoL underwent a redesign to gather evidence of learning for 22 entrustable professional activities (EPAs). Key insights from this development process underscore the importance of the PoL in supporting assessment-for-learning rather than merely assessment-of-learning. This necessitates features for feedback and interaction, ensuring that the PoL functions beyond a mere repository of forms. Additionally, the e-PoL should facilitate triangulation, aggregation, and saturation of data points to effectively measure EPAs. Furthermore, the PoL has not only documented learning but has also played a pivotal role in guiding the development of clinical training by explicitly outlining expectations for both registrars and supervisors. While the initial design and development costs are significant, operational costs become affordable when shared across all training programmes.

Developing an entrustable professional activity for providing health education and consultation in occupational therapy and examining its validity

BackgroundEntrustable Professional Activities (EPA)-based assessment is easily and intuitively used in evaluating the learning outcomes of competency-based medical education (CBME). This study aimed to develop an EPA for occupational therapy focused on providing health education and consultation (TP-EPA3) and examine its validity.MethodsNineteen occupational therapists who had completed online training on the EQual rubric evaluation participated in this study. An expert committee identified six core EPAs for pediatric occupational therapy. TP-EPA3 was developed following the EPA template and refined through consensus meetings. The EQual rubric, a 14-item, five-point criterion-based anchor system, encompassing discrete units of work (DU), entrustable, essential, and important tasks of the profession (EEIT), and curricular role (CR), was used to evaluate the quality of TP-EPA3. Overall scores below 4.07, or scores for DU, EEIT, and CR domains below 4.17. 4.00, and 4.00, respectively, indicate the need for modifications.ResultsThe TP-EPA3 demonstrated good validity, surpassing the required cut-off score with an average overall EQual score of 4.21 (SD = 0.41). Specific domain scores for DU, EEIT, and CR were 3.90 (SD = 0.69), 4.46 (SD = 0.44), and 4.42 (SD = 0.45), respectively. Subsequent revisions clarified observation contexts, enhancing specificity and focus. Further validation of the revised TP-EPA3 and a thorough examination of its reliability and validity are needed.ConclusionThe successful validation of TP-EPA3 suggests its potential as a valid assessment tool in occupational therapy education, offering a structured approach for developing competency in providing health education and consultation. This process model for EPA development and validation can guide occupational therapists in creating tailored EPAs for diverse specialties and settings.

Entrustable Professional Activities in Postgraduate General Surgery Training: A Scoping Review.

To provide an overview of the current use of Entrustable Professional Activities (EPAs) in postgraduate general surgery training internationally. Entrustable Professional Activities (EPAs) were introduced to connect clinical competencies and the professional activities to be entrusted to trainees on graduation. The popularity of EPAs as a framework for assessment is growing globally, including in general surgery. Anecdotally, there appears to be substantial variation in how they are implemented, yet a formal comparison of their use in postgraduate general surgery training is lacking. A scoping review was performed, based on the original five-stage approach described by Arksey and O'Malley with the addition of protocol-specific items from the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols extension for scoping reviews (PRISMA-ScR). Twenty-nine published and grey literature sources were included in the review. Entrustable Professional Activity use in postgraduate general surgery training was identified in 11 unique contexts, including from North America, South America, Europe, Asia, Africa, and Australia. There were substantial differences in the scope and number of EPAs, tools used for EPA assessment, and how EPAs were sequenced through training. Despite wide variation, eight distinct EPAs were common to the majority (>80%) of countries. Several articles described findings of EPA use in postgraduate general surgery training, allowing identification of multiple barriers and facilitators to integration. This review provides guidance for certification and regulatory bodies, program directors, and institutions with ambitions to implement EPAs for assessment and curricular design. In settings where EPAs are already used, the data may facilitate refinement of programs and strategies.

Exploring the Feasibility and Utility of Entrustable Professional Activities in Resident Assessment and Feedback: A Pilot Survey Study

Exploring the Feasibility and Utility of Entrustable Professional Activities in Resident Assessment and Feedback: A Pilot Survey Study

Defining Nurse Practitioner Core Entrustable Professional Activities: Essential Step Toward Competency-Based Education.

Gaps between educational preparation and clinical practice readiness have led to innovative approaches to competence assessment. Entrustable professional activities (EPAs) show promise as a competence assessment framework in graduate nursing education. This study sought to develop and validate a set of EPAs that reflect the core activities performed by all nurse practitioners (NPs). Eight EPAs were developed. A Delphi approach was used to validate the EPAs by NP practice experts located across the United States and representing most NP populations. Consensus was reached after 2 Delphi rounds. The EPAs developed and validated in this study map multiple advanced-level NP competencies to workplace expectations and provide a shared framework for competency-based workplace assessment among NP preceptors from varied health care professions.

Special Report: Entrustable Professional Activities in Sex- and Gender-Based Women’s Health

Special Report: Entrustable Professional Activities in Sex- and Gender-Based Women’s Health

Validation of entrustable professional activities for use in neonatal care residency programs

ObjectiveDefine and develop a set of entrustable professional activities (EPAs) to link clinical training and assessment of the hospital components of neonatal care in neonatology medical residency programs. MethodsAn exploratory study was conducted in two phases using a modified Delphi approach. In the first phase, a committee of five neonatology residency program coordinators drafted an initial set of EPAs based on the national matrix of competencies and on EPAs defined by international organizations. In the second phase, a group of neonatal care physicians and medical residents rated the indispensability and clarity of the EPAs and provided comments and suggestions. ResultsSeven EPAs were drafted by the coordinators´ committee (n = 5) and used in the content validation process with a group (n = 37) of neonatal care physicians and medical residents. In the first Delphi round, all EPAs reached a content validity index (CVI) above 0.8. The coordinators´ committee analyzed comments and suggestions and revised the EPAs. A second Delphi round with the revised EPAs was conducted to validate and all items maintained a CVI above 0.8 for indispensability and clarity. ConclusionSeven entrustable professional activities were developed to assess residents in the hospital components of neonatal care medicine. These EPAs might contribute to implementing competency-based neonatology medical residency programs grounded in core professional activities.

Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process.

Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yetall will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.

Feedback in an Entrustment-Based Objective Structured Clinical Examination: Analysis of Content and Scoring Methods.

Background The integration of entrustable professional activities (EPAs) within objective structured clinical examinations (OSCEs) has yielded a valuable avenue for delivering timely feedback to residents. However, concerns about feedback quality persist. Objective This study aimed to assess the quality and content alignment of verbal feedback provided by examiners during an entrustment-based OSCE. Methods We conducted a progress test OSCE for internal medicine residents in 2022, assessing 7 EPAs. The immediate 2-minute feedback provided by examiners was recorded and analyzed using the Quality of Assessment of Learning (QuAL) score. We also analyzed the degree of alignment with EPA learning objectives: competency milestones and task-specific abilities. In a randomized crossover experiment, we compared the impact of 2 scoring methods used to assess residents' clinical performance (3-point entrustability scales vs task-specific checklists) on feedback quality and alignment. Results Twenty-one examiners provided feedback to 67 residents. The feedback demonstrated high quality (mean QuAL score 4.3 of 5) and significant alignment with the learning objectives of the EPAs. On average, examiners addressed in their feedback 2.5 milestones (61%) and 1.2 task-specific abilities (46%). The scoring methods used had no significant impact on QuAL scores (95% CI -0.3, 0.1, P=.28), alignment with competency milestones (95% CI -0.4, 0.1, P=.13), or alignment with task-specific abilities (95% CI -0.3, 0.1, P=.29). Conclusions In our entrustment-based OSCE, examiners consistently offered valuable feedback aligned with intended learning outcomes. Notably, we explored high-quality feedback and alignment as separate dimensions, finding no significant impact from our 2 scoring methods on either aspect.

Exploring the Use of Natural Language Processing to Understand Emotions of Trainees and Faculty Regarding Entrustable Professional Activity Assessments.

Background In medical education, artificial intelligence techniques such as natural language processing (NLP) are starting to be used to capture and analyze emotions through written text. Objective To explore the application of NLP techniques to understand resident and faculty emotions related to entrustable professional activity (EPA) assessments. Methods Open-ended text data from a survey on emotions toward EPA assessments were analyzed. Respondents were residents and faculty from pediatrics (Peds), general surgery (GS), and emergency medicine (EM), recruited for a larger emotions study in 2023. Participants wrote about their emotions related to receiving/completing EPA assessments. We analyzed the frequency of words rated as positive via a validated sentiment lexicon used in NLP studies. Specifically, we were interested if the count of positive words varied as a function of group membership (faculty, resident), specialty (Peds, GS, EM), gender (man, woman, nonbinary), or visible minority status (yes, no, omit). Results A total of 66 text responses (30 faculty, 36 residents) contained text data useful for sentiment analysis. We analyzed the difference in the count of words categorized as positive across group, specialty, gender, and being a visible minority. Specialty was the only category revealing significant differences via a bootstrapped Poisson regression model with GS responses containing fewer positive words than EM responses. Conclusions By analyzing text data to understand emotions of residents and faculty through an NLP approach, we identified differences in EPA assessment-related emotions of residents versus faculty, and differences across specialties.

Understanding Emotions Impacted by New Assessment Mandates Implemented in Medical Education: A Survey of Residents and Faculty Across Multiple Specialties.

Background Previous research findings show that the overall perception of residents regarding the new entrustable professional activity (EPA) assessment mandates is primarily negative. Hence, this study aims to explore the link between EPA assessment experiences and resident and faculty emotions and expectancy of successfully completing residency training. Methods A standardized questionnaire (Medical Emotions Scale (MES)), which measures 20 unique emotions on a 5-point Likert scale, was used to explore the emotions of residents and faculty members regarding EPA assessments and residents' expectancy of success. Data analysis included descriptive statistics and analysis of variance (ANOVA). Results Ninety-one (N=91) participants (46 faculty members and 45 residents) completed the survey. The results revealed that residents have more negative emotions toward EPA assessments compared to faculty. Additionally, resident and faculty emotions regarding EPA assessments vary across specialty and gender. Conclusions These findings will be crucial in providing the Royal College of Physicians and Surgeons of Canada and medical education programs with concrete evidence and guidance in understanding the perspectives and emotions of residents and faculty towards EPA assessments and residents' beliefs about successfully completing their medical training.

OWIED System to Facilitate the Entrustment Decision in EPA Implementation.

Entrustable professional activities (EPAs) are gaining attention as an approach for adequate implementation of a competency-based evaluation framework in nursing education. This study sought to develop an Online Web-assessment Interface for Entrustment Decision (OWIED) to facilitate the entrustment decision in EPA implementation. A participatory qualitative action research design consisting of two phases was used for this study. The exploration phase was conducted in close collaboration with stakeholders. Following the exploration phase, the primary researcher and a team of subject-matter experts in academic and information technology developed the OWIED system according to the exploration phase results. The necessary features that met the expectations of the stakeholders were identified and assisted in developing the OWIED system. OWIED may provide a valid tool to track and validate nursing student acquisition of core competencies and assist in making entrustment decisions as students fulfill their training requirements in their academic program. [J Nurs Educ. 2024;63(6):356-365.].

Chronicling the Transition to Competency-Based Medical Education in a Small Subspeciality Program.

Background As medical education programs transition to competency-based medical education (CBME), experiences transitioning in the context of small subspecialty programs remain unknown, yet they are needed for effective implementation and continual improvements. Objective To examine faculty and resident experiences transitioning to CBME in a small subspeciality program. Methods Using a qualitative descriptive approach and constructivist lens, faculty and residents in McMaster University's geriatric psychiatry subspecialty program were interviewed about their transition experiences between November 2021 and February 2022, after the program's soft launch of CBME in 2020. Interviews were transcribed and data were analyzed using thematic analysis. Reflexive memo writing and investigator and data triangulation strategies were employed to ensure rigor and trustworthiness of the data. Results Ten of the 17 faculty members (59%) and 3 residents (100%) participated. Six themes were developed: (1) Both faculty and residents see themselves as somewhat knowledgeable about CBME, but sources of knowledge vary; (2) More frequent feedback is beneficial; (3) Aspects of CBME that are challenging for residents are beneficial for faculty; (4) Competence committees are perceived positively despite most participants' limited firsthand experience with them; (5) Small program size is both a barrier and facilitator to providing and receiving feedback; and (6) Suggestions for improvement are centered on helping manage faculty and resident workload imposed by CBME. Conclusions Incongruent expectations surrounding entrustable professional activity management were highlighted as an area requiring support. Collegial relationships among faculty and residents made it difficult for faculty to provide constructive feedback but improved residents' perceptions of the feedback.

Defining a set of teaching EPAs tailored to an undergraduate medical curriculum using a modified Delphi approach

BackgroundThe concept of entrustable professional activities (EPAs) has recently been extended to operationalize professional tasks in teacher training and faculty development in health professions education. The aim of this study is to report on the process and results of defining a set of teaching EPAs (t-EPAs) tailored to the local characteristics of a particular undergraduate medical program.MethodsThe undergraduate medical program at the Charité – Universitätsmedizin Berlin is competency-based, integrates thematic modules and spans 6 years. A writing team identified teaching EPAs based on the program’s study regulations and drafted content descriptions with titles, specifications and knowledge, skills and attitudes. Content validation involved a modified Delphi procedure with a systematic, iterative interaction between a panel of content experts consisting of purposively selected educators and physicians from our faculty (n = 11) and the writing team. The threshold for a consensus was an agreement of 80% of the participants.ResultsAfter two Delphi rounds, a consensus was reached regarding the teaching activities to be included and their content descriptions. The response rate was 100% in both Delphi rounds. The Delphi results include the content descriptions of a total of 13 teaching EPAs, organized into the two overarching EPA domains of classroom-based (n = 10) and workplace-based (n = 3) activities. Tailoring the classroom EPAs to small group teaching and the workplace EPAs to supervising medical students led to several distinct EPAs. Another feature was the development of 2 teaching EPAs for interdisciplinary teaching.ConclusionsIn systematic, Delphi-based process, we defined a set of 13 distinct teaching EPAs tailored to a specific undergraduate medical program that cover the core teaching tasks for faculty in this program. Our report on the principles of the process and the results may guide other medical schools and educators in defining and tailoring teaching EPAs according to their contexts.