Entrustment Decisions Research Articles

Implementing Core Entrustable Professional Activities in Undergraduate Medical Education: A Psychometric Study.

This study examines the feasibility and psychometric results of an assessment of entrustable professional activities (EPAs) as a core component of the clinical program of assessment in undergraduate medical education, assesses the learning curves for each EPA, explores the time to entrustment, and investigates the dependability of the EPA data based on generalizability theory (G theory) analysis. Third-year medical students from the University of Minnesota Medical School in 7 required clerkships from May 2022 through April 2023 were assessed. Students were required to obtain at least 4 EPA assessments per week on average from clinical faculty, residents supervising the students, or assessment and coaching experts. Student ratings were depicted as curves describing their performance over time; regression models were used to fit the curves. The complete class of 240 (138 women [58.0%] and 102 men [42.0%]) third-year medical students at the University of Minnesota Medical School (mean [SD] age at matriculation, 24.2 [2.7] years) participated. There were 32,614 EPA-based assessments (mean [SD], 136 [29.6] assessments per student). Reliability analysis using G theory found that an overall score dependability of 0.75 (range, 0-1) was achieved with 4 assessors on 4 occasions. The desired level of entrustment by academic year end was met by all 240 students (100%) for EPAs 1, 6, and 7, 237 (98.8%), 236 (98.3%), and 218 (90.8%) students for EPAs 2, 5, and 9, respectively, 197 students (82.1%) for EPA 3, 178 students (74.2%) for EPA 4, and 145 students (60.4%) for EPA 12. The most rapid growth was for EPA 2 (β0 = .286), followed by EPA 1 (β0 = .240), EPA 4 (β0 = .236), and EPA 10 (β0 = .230). The study findings suggest that EPA ratings provide reliable and dependable data to make entrustment decisions about students' performance.

Rethinking Informed Consent as an EPA in Surgery.

Over the past decade, entrustable professional activities (EPAs) have become an important element in the competency-based medical education movement. In this Commentary, the authors explore informed consent as an EPA within resident surgical training. In doing so, they foreground the concept of culture and reexamine the nature of trust and entrustment decisions from within a cultural framework. The authors identify role modeling and professional identity formation as core elements in the training process and suggest that faculty are sometimes better off using these tools than uncritically adopting a formal EPA framework for what is, in essence, a professionally oriented and values-based moral enterprise. They conclude that EPAs work best when they are developed at a local level, stressing the unique culture of specialty and program as well as the care that must be taken when attempting to transfer notions of entrustment from the undergraduate medical education level to graduate medical education settings.

OWIED System to Facilitate the Entrustment Decision in EPA Implementation.

Entrustable professional activities (EPAs) are gaining attention as an approach for adequate implementation of a competency-based evaluation framework in nursing education. This study sought to develop an Online Web-assessment Interface for Entrustment Decision (OWIED) to facilitate the entrustment decision in EPA implementation. A participatory qualitative action research design consisting of two phases was used for this study. The exploration phase was conducted in close collaboration with stakeholders. Following the exploration phase, the primary researcher and a team of subject-matter experts in academic and information technology developed the OWIED system according to the exploration phase results. The necessary features that met the expectations of the stakeholders were identified and assisted in developing the OWIED system. OWIED may provide a valid tool to track and validate nursing student acquisition of core competencies and assist in making entrustment decisions as students fulfill their training requirements in their academic program. [J Nurs Educ. 2024;63(6):356-365.].

Development of the entrustable professional activity ‘medication reconciliation’ for clinical pharmacy

BackgroundEntrustable professional activities (EPAs) are observable process descriptions of clinical work units. EPAs support learners and tutors in assessment within healthcare settings. For use amongst our pharmacy students as well as pre-registration pharmacists we wanted to develop and validate an EPA for use in a clinical pharmacy setting at LMU University Hospital.MethodsThe development of the clinical pharmacy EPA followed a set pathway. A rapid literature review informed the first draft, an interprofessional consensus group consisting of pharmacists, nurses, and medical doctors refined this draft. The refined version was then validated via online survey utilising clinical pharmacists from Germany.ResultsWe designed, refined and validated an EPA regarding medication reconciliation for assessment of pharmacy students and trainees within the pharmacy department at LMU University Hospital in Munich. Along with the EPA description an associated checklist to support the entrustment decision was created. For validation an online survey with 27 clinical pharmacists from all over Germany was conducted. Quality testing with the EQual rubric showed a good EPA quality.ConclusionsWe developed the first clinical pharmacy EPA for use in a German context. Medication reconciliation is a suitable EPA candidate as it describes a clinical activity performed by pharmacists in many clinical settings. The newly developed and validated EPA ‘Medication Reconciliation’ will be used to assess pharmacy students and trainees.

Decision-Making Confidence of Clinical Competency Committees for Entrustable Professional Activities

A competency-based assessment framework using entrustable professional activities (EPAs) was endorsed by the American Board of Surgery following a 2-year feasibility pilot study. Pilot study programs' clinical competency committees (CCCs) rated residents on EPA entrustment semiannually using this newly developed assessment tool, but factors associated with their decision-making are not yet known. To identify factors associated with variation in decision-making confidence of CCCs in EPA summative entrustment decisions. This cohort study used deidentified data from the EPA Pilot Study, with participating sites at 28 general surgery residency programs, prospectively collected from July 1, 2018, to June 30, 2020. Data were analyzed from September 27, 2022, to February 15, 2023. Microassessments of resident entrustment for pilot EPAs (gallbladder disease, inguinal hernia, right lower quadrant pain, trauma, and consultation) collected within the course of routine clinical care across four 6-month study cycles. Summative entrustment ratings were then determined by program CCCs for each study cycle. The primary outcome was CCC decision-making confidence rating (high, moderate, slight, or no confidence) for summative entrustment decisions, with a secondary outcome of number of EPA microassessments received per summative entrustment decision. Bivariate tests and mixed-effects regression modeling were used to evaluate factors associated with CCC confidence. Among 565 residents receiving at least 1 EPA microassessment, 1765 summative entrustment decisions were reported. Overall, 72.5% (1279 of 1765) of summative entrustment decisions were made with moderate or high confidence. Confidence ratings increased with increasing mean number of EPA microassessments, with 1.7 (95% CI, 1.4-2.0) at no confidence, 1.9 (95% CI, 1.7-2.1) at slight confidence, 2.9 (95% CI, 2.6-3.2) at moderate confidence, and 4.1 (95% CI, 3.8-4.4) at high confidence. Increasing number of EPA microassessments was associated with increased likelihood of higher CCC confidence for all except 1 EPA phase after controlling for program effects (odds ratio range: 1.21 [95% CI, 1.07-1.37] for intraoperative EPA-4 to 2.93 [95% CI, 1.64-5.85] for postoperative EPA-2); for preoperative EPA-3, there was no association. In this cohort study, the CCC confidence in EPA summative entrustment decisions increased as the number of EPA microassessments increased, and CCCs endorsed moderate to high confidence in most entrustment decisions. These findings provide early validity evidence for this novel assessment framework and may inform program practices as EPAs are implemented nationally.

Responding to the real problem of sustainable resuscitation skills with real assessment. Mixed-methods evaluation of an authentic assessment programme.

The retention of resuscitation skills is a widespread concern, with a rapid decay in competence frequently following training. Meanwhile, training programmes continue to be disconnected with real-world expectations and assessment designs remain in conflict with the evidence for sustainable learning. This study aimed to evaluate a programmatic assessment pedagogy which employed entrustment decision and the principles of authentic and sustainable assessment (SA). We conducted a prospective sequential explanatory mixed methods study to understand and address the sustainable learning challenges faced by final-year undergraduate paramedic students. We introduced a programme of five authentic assessments based on actual resuscitation cases, each integrating contextual elements that featured in these real-life events. The student-tutor consensus assessment (STCA) tool was configured to accommodate an entrustment scale framework. Each test produced dual student led and assessor scores. Students and assessors were surveyed about their experiences with the assessment methodologies and asked to evaluate the programme using the Ottawa Good Assessment Criteria. Eighty-four students participated in five assessments, generating dual assessor-only and student-led results. There was a reported mean score increase of 9% across the five tests and an 18% reduction in borderline or below scores. No statistical significance was observed among the scores from eight assessors across 420 unique tests. The mean student consensus remained above 91% in all 420 tests. Both student and assessor participant groups expressed broad agreement that the Ottawa criteria were well-represented in the design, and they shared their preference for the authentic methodology over traditional approaches. In addition to confirming local sustainability issues, this study has highlighted the validity concerns that exist with conventional resuscitation training designs. We have successfully demonstrated an alternative pedagogy which responds to these concerns, and which embodies the principles of SA, quality in assessment practice, and the real-world expectations of professionals.

Big five personality traits of medical students and workplace performance in the final clerkship year using an EPA framework

BackgroundThe qualities of trainees play a key role in entrustment decisions by clinical supervisors for the assignments of professional tasks and levels of supervision. A recent body of qualitative research has shown that in addition to knowledge and skills, a number of personality traits are relevant in the workplace; however, the relevance of these traits has not been investigated empirically. The aim of this study was to analyse the workplace performance of final-year medical students using an Entrustable Professional Activity (EPA) framework in relation to their personality traits.MethodsMedical students at the end of their final clerkship year were invited to participate in an online survey-based, cross-sectional field study. In the survey, the workplace performance was captured using a framework consisting of levels of experienced supervision and a defined set of 12 end-of-undergraduate medical training EPAs. The Big Five personality traits (extraversion, agreeableness, conscientiousness, neuroticism, and openness) of the participating medical students were measured using the Big Five Inventory-SOEP (BFI-S), which consists of 15 items that are rated on a seven-point Likert scale. The data were analysed using descriptive and inferential statistics.ResultsThe study included 880 final-year medical students (mean age: 27.2 years, SD = 3.0; 65% female). The levels of supervision under which the final-year clerkship students carried out the EPAs varied considerably. Significant correlations were found between the levels of experienced supervision and all Big Five dimensions The correlations with the dimensions of extraversion, agreeableness, conscientiousness and openness were positive, and that for the neuroticism dimension was negative (range r = 0.17 to r = − 0.23). Multiple regression analyses showed that the combination of the Big Five personality traits accounted for 0.8–7.5% of the variance in supervision levels on individual EPAs.ConclusionsUsing the BFI-S, we found that the levels of supervision on a set of end-of-undergraduate medical training EPAs were related to the personality traits of final-year medical students. The results of this study confirm the existing body of research on the role of conscientiousness and extraversion in entrustment decision-making and, in particular, add the personality trait of neuroticism as a new and relevant trainee quality to be considered.

Clinical supervisors’ experience of a first-time application of entrustable professional activities in clinical supervision of medical students: findings from a Swedish pilot study

BackgroundRecently, all medical universities in Sweden jointly developed a framework for Entrustable Professional Activities (EPAs) for work-based training and assessment. This framework is now being introduced nationally in the new 6-year undergraduate medical programme that directly lead to a licence to practise. When EPAs are introduced, it is of central importance to gain clinical supervisors’ acceptance to apply the framework in their supervision of students. The aim of this study was therefore to investigate how clinical supervisors, not familiar with EPAs, experience clinical supervision using the framework for EPAs.MethodsWe used a purposive sampling to recruit clinical supervisors. They were given written information on EPAs with a selection of suitable EPAs and the Swedish observation rating scale for assessment of autonomy, and they were offered to attend a 30-minute introductory web course. The participants were informed that EPAs were to be tested, and the students were asked to participate. After the study period the clinical supervisors participated in semi-structured interviews. Inductive qualitative content analysis was used to analyse the transcribed interviews.ResultsThree general themes emerged in the qualitative analysis: Promoting Feedback, Trusting Assessments and Engaging Stakeholders. The participants described benefits from using EPAs, but pointed out a need for preparation and adaptation to facilitate implementation. The structure was perceived to provide structured support for feedback, student involvement, entrustment decisions, enabling supervisors to allow the students to do more things independently, although some expressed caution to rely on others’ assessments. Another concern was whether assessments of EPAs would be perceived as a form of examination, steeling focus from formative feedback. To understand the concept of EPA, the short web-based course and written information was regarded as sufficient. However, concern was expressed whether EPA could be applied by all clinical supervisors. Involvement and adaption of the workplace was pointed out as important since more frequent observation and feedback, with documentation requirements, increase the time required for supervision.ConclusionsEPAs were accepted as beneficial, promoting structured feedback and assessments of the students’ autonomy. Preparation of supervisors and students as well as involvement and adaptation of the workplace was pointed out as important.

Validity of Entrustable Professional Activities in a National Sample of General Surgery Residency Programs.

The American Board of Surgery has endorsed competency-based education as vital to the assessment of surgical training. From 2018 to 2020, a national pilot study was conducted at 28 general surgery programs to evaluate feasibility of implementing entrustable professional activities (EPAs) for 5 common general surgical conditions. ACGME core competency Milestones were also rated for each resident by program clinical competency committees. This study aimed to evaluate the validity of general surgery EPAs compared with Milestones. Prospectively collected, de-identified EPA Pilot Study data were analyzed. EPAs studied were EPA-1 (gallbladder), EPA-2 (inguinal hernia), EPA-3 (right lower quadrant pain), EPA-4 (trauma), and EPA-5 (consult). Variables abstracted included levels of EPA entrustment (1 to 5) and corresponding ACGME Milestone subcompetency ratings (1 to 5) for the same study cycle. Spearman's correlations were used to evaluate the relationship between summative EPA scores and corresponding Milestone ratings. A total of 493 unique residents received a summative entrustment decision. EPA summative entrustment scores had moderate-to-strong positive correlation with mapped Milestone subcompetencies, with median rho value of 0.703. Among operation-focused EPAs, median rho values were similar between EPA-1 (0.688) and EPA-2 (0.661), but higher for EPA-3 (0.833). EPA-4 showed a strong positive correlation with diagnosis and communication competencies (0.724), whereas EPA-5, mapped to the most Milestone subcompetencies, had the lowest median rho value (0.455). Moderate-to-strong positive correlation was noted between EPAs and patient care, medical knowledge, and communication Milestones. These findings support the validity of EPAs in general surgery and suggest that EPA assessments can be used to inform Milestone ratings by clinical competency committees.

Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias in feedback

The entrustment framework redirects assessment from considering only trainees’ competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic characteristics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differentially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors’ themes predominantly reflected skills related to patient presentations, while trainees’ themes were broader—including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment, p < 0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1–4 scale, p < 0.05). We also found biases tied to demographic characteristics: trainees’ documentation reflected more positive sentiment in the case of male trainees (+ 1.3%, p < 0.05) and of trainees underrepresented in medicine (UIM) (+ 1.3%, p < 0.05). Entrustment ratings did not appear to reflect these biases, neither when documented by trainee nor supervisor. As such, bias appeared to influence the emotive language trainees used to document entrustment more than the degree of entrustment they experienced. Mitigating these biases is nonetheless important because they may affect trainees’ assimilation into their roles and formation of trusting relationships.

Next steps for workplace-based assessments of entrustable professional activities.

Florida International University Herbert Wertheim College of Medicine (FIU-HWCOM) participated in the AAMC Core Entrustable Professional Activities (EPA) implementation pilot. Entrustment decision processes based on data from workplace-based assessments (WBAs) were piloted. Outcomes illustrated challenges including variability across EPAs with regards to learner level alignment and feasibility of data collection in the form of WBAs. In addition, students reported discomfort requesting WBA completion by preceptors and dissatisfaction with associated feedback. To guide future directions, we conducted a survey of third-year students to better understand their experience with and perceptions of WBAs used to evaluate EPAs at FIU-HWCOM. Survey response was 96% (n= 107/112). Most (84%) reported that WBAs were not valuable to their development and that preceptors often did not complete WBAs in a timely fashion. Many (47%) reported not receiving verbal feedback. Most students (78%) used language in written responses demonstrating confusion between the EPAs and the WBAs used to assess them. The use of WBAs to assess EPAs did not have its intended impact at FIU-HWCOM. For future classes, WBA forms will consist of paper cards with questions directly assessing performance of skills aligned with EPAs 1, 5 and 6 only. To continue to promote feedback, students will be required to collect WBAs on all clerkships, but the number of required WBAs will be less than prior and no entrustment decisions will be made.

Do summative entrustment decisions actually lead to entrustment?

Entrustable professional activities (EPAs) were introduced across Dutch postgraduate programmes between 2017 and 2019. We aimed to understand the extent to which residents actually were granted increased clinical responsibility upon receiving summative entrustment for an EPA, a critical feature of its use. A survey study was conducted among all Dutch residents who started dermatology training in 2018 and 2019 and all Dutch dermatology programme directors (PDs). We chose an EPA designed for early entrustment in residency (identification, treatment and care regarding a simple dermatological problem in the ambulatory setting). The survey contained two hypothetical clinical cases that aligned with this EPA. The questions were aimed to determine whether and when residents should request supervision. Similar questions were posed to PDs. Twenty four residents (56%) and 19 PDs (79%) completed the survey. The majority of the residents (65%) and PDs (63%) confirmed that competent dermatology residents (level 4) are generally allowed to perform EPA1 unsupervised, particularly when seeing patients from GPs. However, still a substantial proportion of the level 4 residents, working in University Medical Centers (36%) indicated that they had to request supervision in the assessment of these patients. For 2nd opinions, the results were typically the opposite. This study demonstrated that, at least in one specialty and one country, the introduction of EPAs and entrustment decision making procedure generally led to the intended autonomy of the resident.

How well do workplace-based assessments support summative entrustment decisions? A multi-institutional generalisability study.

Assessment of the Core Entrustable Professional Activities for Entering Residency requires direct observation through workplace-based assessments (WBAs). Single-institution studies have demonstrated mixed findings regarding the reliability of WBAs developed to measure student progression towards entrustment. Factors such as faculty development, rater engagement and scale selection have been suggested to improve reliability. The purpose of this investigation was to conduct a multi-institutional generalisability study to determine the influence of specific factors on reliability of WBAs. The authors analysed WBA data obtained for clerkship-level students across seven institutions from 2018 to 2020. Institutions implemented a variety of strategies including selection of designated assessors, altered scales and different EPAs. Data were aggregated by these factors. Generalisability theory was then used to examine the internal structure validity evidence of the data. An unbalanced cross-classified random-effects model was used to decompose variance components. A phi coefficient of >0.7 was used as threshold for acceptable reliability. Data from 53 565 WBAs were analysed, and a total of 77 generalisability studies were performed. Most data came from EPAs 1 (n = 17 118, 32%) 2 (n = 10 237, 19.1%), and 6 (n = 6000, 18.5%). Low variance attributed to the learner (<10%) was found for most (59/77, 76%) analyses, resulting in a relatively large number of observations required for reasonable reliability (range = 3 to >560, median = 60). Factors such as DA, scale or EPA were not consistently associated with improved reliability. The results from this study describe relatively low reliability in the WBAs obtained across seven sites. Generalisability for these instruments may be less dependent on factors such as faculty development, rater engagement or scale selection. When used for formative feedback, data from these instruments may be useful. However, such instruments do not consistently provide reasonable reliability to justify their use in high-stakes summative entrustment decisions.

A realist synthesis of prospective entrustment decision making by entrustment or clinical competency committees.

The real-world mechanisms underlying prospective entrustment decision making (PEDM) by entrustment or clinical competency committees (E/CCCs) are poorly understood. To advance understanding in this area, the authors conducted a realist synthesis of the published literature to address the following research question: In E/CCC efforts to make defensible prospective entrustment decisions (PEDs), what works, for whom, under what circumstances and why? Realist work seeks to understand the contexts (C), mechanisms (M) and outcomes (O) that explain how and why things work (or do not). In the authors' study, contexts included individual E/CCC members, E/CCC structures and processes, and training programmes. The outcome (i.e. desired outcome) was a PED. Mechanisms were a substantial focus of the analysis and informed the core findings. To define a final corpus of 52 included papers, the authors searched four databases, screened all results from those searches and performed a full-text review of a subset of screened papers. Data extraction focused on developing context-mechanism-outcome configurations from the papers, which were used to create a theory for how PEDM leads to PEDs. PEDM is often driven by default (non-deliberate) decision making rather than a deliberate process of deciding whether a trainee should be entrusted or not. When defaulting, some E/CCCs find red flags that sometimes lead to being more deliberate with decision making. E/CCCs that seek to be deliberate describe PEDM that can be effortful (when data are insufficient or incongruent) or effortless (when data are robust and tell a congruent story about a trainee). Both information about trainee trustworthiness and the sufficiency of data about trainee performance influence PEDM. Several moderators influence what is considered to be sufficient data, how trustworthiness data are viewed and how PEDM is carried out. These include perceived consequences and associated risks, E/CCC member trust propensity, E/CCC member personal knowledge of and experience with trainees and E/CCC structures and processes. PEDM is rarely deliberate but should be. Data about trainee trustworthiness are foundational to making PEDs. Bias, equity and fairness are nearly absent from the papers in this synthesis, and future efforts must seek to advance understanding and practice regarding the roles of bias, equity and fairness in PEDM.

Summative Entrustment Decisions in UME: Outcomes of a Binding Entrustment Committee Decision.

Summative Entrustment Decisions in UME: Outcomes of a Binding Entrustment Committee Decision.

Entrustment Decision Making in the Intensive Care Unit: It's About More Than the Learner.

The provision of graded supervision affording progressive autonomy is fundamental to the progression of a medical learner toward competency for independent practice; the decision of how much supervision versus autonomy to provide a trainee in the execution of clinical care constitutes an entrustment decision. Despite entrustment decision making occurring both daily in practice and summatively at points of matriculation through stages of medical training, the factors influencing entrustment decisions remain poorly understood across clinical contexts. This study was designed to explore the central research question: How are entrustment decisions made in the medical intensive care unit (ICU)? This qualitative case study used semistructured interviews with attending pulmonary and critical care physicians in the medical ICU at a major midwestern medical center to explore the entrustment decision-making process as it was enacted in the clinical environment. Five major themes emerged from the data: 1) task, circumstance, and trainee factors contribute to entrustment decision making; 2) ad hoc entrustment decisions are enacted by supervisors with a consideration of the care team as a unit, not only an individual; 3) autonomy does not only arise out of entrustment, but outcomes of prior autonomous actions by the trainee inform the intention to entrust; 4) entrustment decision making includes a social process of back-and-forth akin to negotiation; and 5) entrustment is a learned skill. The process of entrustment decision making in the ICU is more complex than prior frameworks have captured; a model with more complete incorporation of the factors that influence entrustment in the ICU is presented. It is not clear how often ad hoc entrustment decisions in clinical practice are primarily driven by factors pertaining directly to trainee competence, which carries implications in the use of entrustment for assessment.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment

BackgroundEntrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one’s abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment.ObjectivesWe hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME).MethodsIn fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale.ResultsOne-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59–0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05).ConclusionsWe found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Development and validation of core entrustable professional activities for abdominal radiology

ObjectivesTo develop and validate European entrustable professional activities (EPAs) for sub-specialised hepatobiliary and gastrointestinal (HB/GI) diagnostic imaging.Materials and methodsBoth European Society of Radiology and national curricula in HB/GI diagnostic radiology were thoroughly reviewed, resulting in preliminary EPAs drafted by a pilot group of expert radiologists in 2 different countries. Each EPA was fully described with 7 components (Specification/limitations; Potential risks of failing; Relevant domains of competence; Required experience, knowledge, skills, attitude and behaviour; Assessment information sources to assess progress and ground a summative entrustment decision; Entrustment for which level of supervision is to be reached; and Expiration date). The modified Delphi method with 3 Delphi rounds was chosen for validation. Content validity index (CVI) and median values were used for validation.ResultsThere were 15 preliminary EPAs, some of them divided according to 2 levels: resident and fellow level. The 37 members of the Delphi group were based in 2 different European countries with a background experience of 10 represented countries. Subsequent to the first Delphi round, 6 EPAs were accepted (CVI ≥ 0.8, median ≥ 4), 6 needed major revisions (CVI 0.7–0.79, median ≥ 4), 3 were rejected (CVI < 0.7) and 1 was added. After the second Delphi round, both the 6 revised EPAs and the additional one met the validation criteria (CVI ≥ 0.8, median ≥ 4). Finally, 13 EPAs were validated during the 3rd Delphi round with an agreement percentage of 95–100%.ConclusionThis study creates and validates EPAs for sub-specialised HB/GI diagnostic imaging.Critical relevance statementThirteen EPAs for sub-specialised hepatobiliary and gastrointestinal diagnostic imaging were created with a strong methodology, and as a first example set in sub-specialised diagnostic imaging, they provide a template for others to be created.Key points• The competence-based teaching in medical studies has recently been reintroduced through EPAs.• Thirteen EPAs have been developed for hepatobiliary and gastrointestinal sub-specialised diagnostic imaging.• These EPAs were validated using a Delphi modified method and provide a template for other to be created.Graphical abstract

A pilot study evaluating the feasibility of assessing undergraduate pharmacy and medical students interprofessional collaboration during an online interprofessional education intervention about hospital discharge

BackgroundInterprofessional education (IPE) has been identified as a strategy towards improving competence at interprofessional working and collaboration within teams. Entrustable professional activities (EPAs) provide a framework for translating competencies into elements of clinical practice, some of which in healthcare are inherently interprofessional. However, it is challenging to reconcile that entrustment decisions about student competence in an interprofessional activity are made about an individual without considering the dynamics and tensions between interprofessional team members and the task itself. This can influence students’ development and demonstration of competence at interprofessional collaboration.MethodsIn this study, undergraduate medical and pharmacy students worked in pairs online (Zoom) to undertake the hospital discharge process (a professional activity reliant on interprofessional collaboration) for a simulated patient, producing a hospital discharge letter and completing a consultation with the simulated patient. The online sessions were recorded and interprofessional behaviours were assessed using a validated scale completed by an interprofessional assessment team. Students undertook this IPE intervention three times after receiving feedback and a period of reflection each time.ResultsEighteen students participated across the entire intervention and 27 one-hour online IPE sessions were completed and recorded. Students demonstrated statistically significant improvements in interprofessional behaviours across the three iterations (p < 0.05 for all the sessions). The discharge letter students produced also improved over the three sessions (p = 0.01). Students found the educational sessions useful and relevant.ConclusionThis online IPE intervention provided the students with an authentic opportunity to work collaboratively. At the end of each iteration, students received feedback about their work as a team and about the discharge letter, helping students to reflect and purposefully develop their performance. The IPE intervention with this assessment strategy is feasible and allows student development to be captured but has proved to be time and resource intensive.

Enhancing the Value of Surgical Entrustable Professional Activities through Integrative Learning Analytics

To demonstrate the value of integrating surgical resident Entrustable Professional Activity (EPA) data into a learning analytics platform that provides meaningful feedback for formative and summative decision-making. Description of the Surgical Entrustable Professional Activities (SEPA) analytics dashboard, and examples of summary analytics and intuitive display features. Department of Surgery, University of Wisconsin Hospital and Clinics. Surgery residents, faculty, and residency program administrators. We outline the major functionalities of the SEPA dashboard and offer concrete examples of how these features are utilized by various stakeholders to support progressive entrustment decisions for surgical residents. Our intuitive analytics platform allows for seamless integration of SEPA microassessment data to support Clinical Competency Committee (CCC) decisions for resident evaluation and provides point of training feedback to faculty and trainees in support of progressive autonomy.