Accreditation Council For Graduate Medical Education Milestones Research Articles

Using ACGME milestones as a formative assessment for the internal medicine clerkship: a consecutive two-year outcome and follow-up after graduation

BackgroundThis study evaluated the utility of using Accreditation Council for Graduate Medical Education (ACGME) Milestones as a formative assessment tool for the fifth- and sixth-grade medical students’ performance in their internal medicine (IM) clerkship and the same students’ performance in their post-graduate year (PGY) IM training.MethodsRetrospective data were collected from 65 medical students completing the two-year IM clerkship in the academic years 2019 and 2020 and 26 of the above students completing their PGY-1 training at the same university hospital in the academic year 2021. Data included the assessment results of 7 of the ACGME IM Milestones, information on admitted patients assigned to the students, and surveys of the students’ satisfaction.ResultsThe analysis included 390 assessment results during the IM clerkship and 78 assessment results during the PGY-1 training. Clinical teachers commonly rated level 3 to medical students in the IM clerkship, with PC-2 subcompetency receiving the lowest rating among seven subcompetencies. The levels of most subcompetencies showed stationary in the two-year IM clerkship. Significant improvement was observed in all subcompetencies during the PGY-1 training. The medical students in the second-year IM clerkship expressed higher satisfaction with implementing Milestones than in their first-year IM clerkship and perceived Milestones assessments’ usefulness as learning feedback.ConclusionsUsing ACGME Milestones as a formative assessment tool in the IM clerkship yielded promising outcomes. Longitudinal follow-up of subcompetencies facilitated tracking students’ development and providing constructive feedback.

Assessment of Agenda Setting With ACGME Milestones in Family Medicine Residents.

Family medicine residents are scored via milestones created by the Accreditation Council for Graduate Medical Education (ACGME) on various clinical domains, including communication. Communication involves a resident's ability to set an agenda, but this is rarely taught in formal education. Our study aimed to examine the relationship between ACGME Milestone achievement and ability to set a visit agenda, as measured by direct observation (DO) forms. We examined biannual (December, June) ACGME scores for family medicine residents at an academic institution from 2015-2020. Using faculty DO scores, we rated residents on six items corresponding to agenda setting. We used Spearman and Pearson correlations and two-sample paired t tests to analyze results. We analyzed a total of 246 ACGME scores and 215 DO forms. For first-year residents, we found significant, positive associations between agenda-setting and the total Milestone score (r[190]=.15, P=.034) in December, and in individual (r[190]=.17, P=.020) and total communication scores (r[186]=.16, P=.031), in June. However, for first-year residents, we found no significant correlations with communication scores in December or in the total milestone scores in June. We found significant progression for consecutive years in both communication milestones (t=-15.06, P<0.001) and agenda setting (t=-12.26, P<.001). The significant associations found in agenda setting with both ACGME total communication and Milestone scores for first-year residents only, suggests that agenda setting may be fundamental in early resident education.

Training Generations of Clinician Educators: Applying the Novel Clinician Educator Milestones to Faculty Development

Training Generations of Clinician Educators: Applying the Novel Clinician Educator Milestones to Faculty Development

Emergency medical services Milestones 2.0: What has changed?

Since 2015, development of competencies by emergency medical services (EMS) fellows have been evaluated using the EMS Milestones 1.0 developed by a working group consisting of relevant stakeholders convened by the Accreditation Council for Graduate Medical Education (ACGME). Feedback from users and data collected from the milestones assessments in the interim indicated a need for revision of the original milestones. In May 2021, the Milestones 2.0 working group was convened for the purpose of revising this specialty-specific assessment tool. A working group consisting of representatives from American Board of Emergency Medicine, the Review Committee for Emergency Medicine, and volunteers selected by the ACGME Milestones Committee, chaired by the ACGME vice president for milestones development, was convened using a virtual platform to revise the milestones and develop a supplemental guide for use along with the Milestones 2.0. There were no in-person meetings of this working group due to the COVID-19 pandemic. Data from milestones reporting, discussion within the working group, stakeholder input, and public commentary were used to revise the original milestones. A new supplemental guide to enhance milestone usability and provide recommended resource materials was also developed for use alongside the milestones. The EMS Milestones 2.0 and accompanying supplemental guide provide an updated framework for fellowship programs to use as a guide for developing the competencies necessary for independent practice as EMS physicians and in the formal, competency-based evaluation of trainees as required by the ACGME.

Development of Consensus Guidelines for Hematology Fellow Competency in Peripheral Blood Smear Review Utilizing Nominal Group Technique

INTRODUCTION: Graduate medical education (GME) training in hematology in the United States is accredited by the Accreditation Council for Graduate Medical Education (ACGME). Trainees provide patient care that requires manual review and interpretation of peripheral blood smears (PBS). The ACGME milestones were developed with significant input from the American Society of Hematology (ASH) and mandate fellow competency in the interpretation of peripheral blood smears. However, there are currently no publications that define competency or propose consensus recommendations (CRs) for PBS review. The primary aim of this study was to establish CRs on PBS review with a focus at the GME level. METHODS: Nominal group technique (NGT) and constructivist grounded theory (CGT) were selected given the lack of existing published theory and consensus on the methodology of PBS review or other analogous visuospatial skills within GME. A convenience sample of hematologists with expertise in training and assessing fellows was recruited through the ASH Medical Educators Institute. Goal accrual was 5-10 hematologists. Participants were required to regularly review PBSs as part of their clinical practice and attend two 1-hour virtual focus group sessions. The facilitator guide for session 1 was developed a priori to prompt open discussion on PBS review. Dialogue was transcribed and organized into thematic concepts through inductive coding. Data from session 1 were analyzed using a mixed methods approach and were used to develop a facilitator guide for session 2. Draft CRs were created using CGT. Weak and strong consensus were pre-specified as 50 - 69%, and >70% agreement, respectively. CRs underwent iterative review which would continue for a pre-specified minimum of 2, but no more than 3 rounds. RESULTS: Eight academic hematologists (5 men, 3 women) enrolled in the focus group (Table 1). Duration of clinical practice ranged from 5 to 33 years. Clinical focus spanned classical and malignant hematologic conditions. Half had formally instructed trainees in course work on PBS morphology. All routinely worked with fellows. CRs were generated through CGT. Three rounds of review occurred. Participants provided a statement of agree vs. disagree and offered qualitative suggestions for each statement in which they disagreed. Response rates per round were 87.5%, 87.5%, and 100%. The most commonly identified themes included recognition of morphology suggestive of thrombotic microangiopathies and distinguishing between malignant vs. benign white blood cells (WBC), specifically as it pertains to acute leukemias. Strong consensus was reached on multiple aspects of PBS education, including curricular considerations, method of review, and morphology (Table 2). All agreed that trainees should learn PBS review through a systematic approach, although some agreed that this systematic approach is not always used in clinical practice. The group agreed that trainees should be educated through PBS review of patients under their clinical care as well as through formal slide "libraries". Many discussed the changing landscape of PBS education with the advent of digital microscopy, and formed strong consensus that trainees should understand the benefits and limitations of digital microscopy. Group discussion largely focused on disorders of WBCs and red blood cells (RBC) (43% and 48% of all coded statements respectively). Most discussion focused on disorders in which prompt recognition was imperative to avert significant patient morbidity or mortality. However, all agreed that trainees must also be competent in the identification of common conditions, regardless of the degree of clinical importance, including normal peripheral blood smear morphology. The group was unable to reach consensus on how to balance these competing philosophies. CONCLUSIONS: A formal theory of PBS education was systematically developed using CGT. Experts were able to form strong CRs on almost every aspect of PBS review at the GME level. FUTURE DIRECTIONS: We propose that these guidelines serve as a template for future research and curricular development for both PBS review and other visuospatial tasks in medical education. This framework can also be used by organizations, such as ACGME and ASH, when formally reviewing and codifying guidelines on PBS review. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal

Beyond opioid prescribing: Evaluation of a substance use disorder curriculum for OBGYN residents.

Amidst the current opioid crisis, there is a need for better integration of substance use disorder screening and treatment across specialties. However, there is no consensus regarding how to best instruct OBGYN trainees in the clinical skills related to opioid and other substance use disorders (SUD). Study objectives were (1) to assess the effectiveness a SUD curriculum to improve self-reported competence among OBGYN residents and (2) to explore its effectiveness to improve attending evaluations of residents' clinical skills as well as its feasibility and acceptability from the resident perspective. A pilot 3-session curriculum was developed and adapted to SUD screening and treatment which included readings, didactics, and supervised outpatient clinical experiences for OBGYN post-graduate year 1 (PGY-1) residents rotating through an integrated OBGYN-SUD clinic. Eighteen residents completed pre and post clinical skills self-assessments (SUD screening, counseling, referring, Motivational Interviewing) using an adapted Zwisch Rating Scale (range 1-5). Scores were compared between time points using paired t-tests. Sub-samples also (a) were evaluated by the attending on three relevant Accreditation Council for Graduate Medical Education Milestones (ACGME) milestone sets using the web-based feedback program, myTIPreport (n = 10) and (b) completed a qualitative interview (n = 4). All PGY-1s (18/18) across three academic years completed the 3-session SUD curriculum. Clinical skill self-assessments improved significantly in all areas [SUD Screening (2.44 (0.98) vs 3.56 (0.62), p = <0.01); Counseling (1.81 (0.71) vs 3.56 (0.51), p = < .01; Referring (2.03 (0.74) vs 3.17 (0.71), p = < .01; Motivational Interviewing (1.94 (1.06) vs 3.33 (0.69), p = < .01)]. Milestone set levels assigned by attending evaluations (n = 10) also improved. Qualitative data (n = 4) revealed high acceptability; all curriculum components were viewed positively, and feedback was provided (e.g., desire for more patient exposures). A pilot SUD curriculum tailored for OBGYN PGY-1 residents that goes beyond opioid prescribing to encompass SUD management is feasible, acceptable and likely effective at improving SUD core clinical skills.

Telehealth Competencies in Medical Education: New Frontiers in Faculty Development and Learner Assessments.

Telehealth visits have become an integral model of healthcare delivery since the COVID-19 pandemic. This rapid expansion of telehealthcare delivery has forced faculty development and trainee education in telehealth to occur simultaneously. In response, academic medical institutions have quickly implemented clinical training to teach digital health skills to providers across the medical education continuum. Yet, learners of all levels must still receive continual assessment and feedback on their skills to align with the telehealth competencies and milestones set forth by the Association of American Medical Colleges (AAMC) and the Accreditation Council for Graduate Medical Education (ACGME). This paper discusses key educational needs and emerging areas for faculty development in telehealth teaching and assessment of telehealth competencies. It proposes strategies for the successful integration of the AAMC telehealth competencies and ACGME milestones into medical education, including skills in communication, data gathering, and patient safety with appropriate telehealth use. Direct observation tools in the paper offer educators novel instruments to assess telehealth competencies in medical students, residents, and peer faculty. The integration of AAMC and ACGME telehealth competencies and the new assessment tools in this paper provide a unique perspective to advance clinical practice and teaching skills in telehealthcare delivery.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11606-022-07564-8.

Diversity, Equity, and Inclusion Milestones: Creation of a Tool to Evaluate Graduate Medical Education Programs.

As the Accreditation Council for Graduate Medical Education (ACGME) began to ask programs to report their efforts surrounding diversity, equity, and inclusion (DEI), program directors felt ill prepared to evaluate their programs and measure change. To develop a tool that would allow graduate medical education (GME) programs to evaluate the current state of DEI within their residencies, identify areas of need, and track progress; to evaluate feasibility of using this assessment method within family medicine training programs; and to analyze and report pilot data from implementation of these milestones within family medicine residency programs. The Association of Family Medicine Residency Directors (AFMRD) Diversity and Health Equity (DHE) Task Force developed a tool for program DEI evaluation modeled after the ACGME Milestones. These milestones focus on DEI assessment in 5 key domains: Institution, Curriculum, Evaluation, Resident Personnel, and Faculty Personnel. After finalizing a draft, a pilot implementation of the milestones was conducted by a convenience sample of 10 AFMRD DHE Task Force members for their own programs. Scores varied widely across surveyed programs for all milestones. Highest average scores were seen for the Curriculum milestone (2.65) and the lowest for the Faculty Personnel milestone (2.0). Milestone assessments were completed within 10 to 40 minutes using various methods. The AFMRD DEI Milestones were developed for program assessment, goal setting, and tracking of progress related to DEI within residency programs. The pilot implementation showed these milestones were easily used by family medicine faculty members in diverse settings.

Laboratory Management for Cytopathology Fellows: Our Institution’s Curriculum

Abstract Introduction/Objective The cytopathology laboratory is one of the most highly regulated clinical laboratories. The Accreditation Council for Graduate Medical Education (ACGME) milestones include competencies in laboratory management. Additionally, many cytology fellows will be involved in laboratory management after finishing training. Therefore, it is imperative to incorporate formal laboratory management training for cytology fellows. Our cytopathology department implemented a formal laboratory management curriculum this year, with emphasis placed on the milestones (1.0 and 2.0) provided by the ACGME. Methods/Case Report The curriculum is divided into seven modules extending throughout the fellowship year (August-April). The modules include hand-off communication, process improvement, laboratory accreditation (introduction, intermediate, advanced) and error management (introduction, intermediate). Each module has two components: self-study of assigned reading materials and online courses, followed by an interactive session with faculty and/or laboratory personnel. Several engaging, high-quality resources are used including Institute for Healthcare Improvement Open School, modules from professional organizations, PubMed articles, National Quality Forum, departmental committees and mock inspections. Each module is followed by an interactive session to review policies, procedures and/or checklists. The fellows also spend time in the cytology laboratory preparation area and participate in Quality Improvement meetings. Results (if a Case Study enter NA) The fellowship director discusses the educational expectations at the start of each module. Evidence of compliance is demonstrated by uploading all of the completed assignments to MedHub (a web- based training management system) at the end of the module. All of the modules include self-assessment questions that are reviewed by the fellowship director. Conclusion Our cytopathology laboratory introduced a structured laboratory management curriculum. This curriculum fulfills some of the laboratory management competencies put forth in the ACGME milestones. This curriculum will also help to prepare fellows to confidently participate in cytology laboratory management as they enter the workforce.

ACGME Milestones in global health: Need for standardized assessment of global health training in obstetrics/gynecology residency.

Collective interest in global health training during US obstetrics/gynecology (Ob/Gyn) residency has grown over the past decade. The benefits of participation in global health electives have been well described. This review seeks to determine what literature exists regarding the use of Accreditation Council for Graduate Medical Education (ACGME) Milestones in Ob/Gyn residency as an assessment tool to evaluate global health programs. The PubMed database was searched from July 14, 2020 to August 20, 2021, using six search phrases: "global health curriculum(s) and ACGME"; "international health and ACGME"; "global health and Ob/Gyn residency"; "international health and Ob/Gyn residency"; "global health and Ob/Gyn residents"; and "global health curriculum(s) and Ob/Gyn residency." Publications that described global health programming outside of residency, within other medical specialties, and/or at non-US institutions were excluded from this review. In total, 259 publications resulted from the preliminary search. Five articles described US global health residency training in Ob/Gyn in some capacity. Only one publication described a specific global health elective and its evaluation with respect to ACGME Milestones. Despite growing popularity of global health electives among residency programs, few are assessing the educational value of these offerings using ACGME Milestones or describing these efforts in the literature.

Using Learning Analytics to Examine Achievement of Graduation Targets for Systems-Based Practice and Practice-Based Learning and Improvement: A National Cohort of Vascular Surgery Fellows

Surgeons provide patient care in complex health care systems and must be able to participate in improving both personal performance and the performance of the system. The Accreditation Council for Graduate Medical Education (ACGME) Vascular Surgery Milestones are utilized to assess vascular surgery fellows' (VSF) achievement of graduation targets in the competencies of Systems Based Practice (SBP) and Practice Based Learning and Improvement (PBLI). We investigate the predictive value of semiannual milestones ratings for final achievement within these competencies at the time of graduation. National ACGME milestones data were utilized for analysis. All trainees entering the 2-year vascular surgery fellowship programs in July 2016 were included in the analysis (n = 122). Predictive probability values (PPVs) were obtained for each SBP and PBLI sub-competencies by biannual review periods, to estimate the probability of VSFs not reaching the recommended graduation target based on their previous milestones ratings. The rate of nonachievement of the graduation target level 4.0 on the SBP and PBLI sub-competencies at the time of graduation for VSFs was 13.1-25.4%. At the first time point of assessment, 6 months into the fellowship program, the PPV of the SBP and PBLI milestones for nonachievement of level 4.0 upon graduation ranged from 16.3-60.2%. Six months prior to graduation, the PPVs across the 6 sub-competencies ranged from 14.6-82.9%. A significant percentage of VSFs do not achieve the ACGME Vascular Surgery Milestone targets for graduation in the competencies of SBP and PBLI, suggesting a need to improve curricula and assessment strategies in these domains across vascular surgery fellowship programs. Reported milestones levels across all time point are predictive of ultimate achievement upon graduation and should be utilized to provide targeted feedback and individualized learning plans to ensure graduates are prepared to engage in personal and health care system improvement once in unsupervised practice.

Association Between Internal Medicine Residency Applicant Characteristics and Performance on ACGME Milestones During Intern Year.

Residency programs apply varying criteria to the resident selection process. However, it is unclear which applicant characteristics reflect preparedness for residency. We determined the applicant characteristics associated with first-year performance in internal medicine residency as assessed by performance on Accreditation Council for Graduate Medical Education (ACGME) Milestones. We examined the association between applicant characteristics and performance on ACGME Milestones during intern year for individuals entering Northwestern University's internal medicine residency between 2013 and 2018. We used bivariate analysis and a multivariable linear regression model to determine the association between individual factors and Milestone performance. Of 203 eligible residents, 198 (98%) were included in the final sample. One hundred fourteen residents (58%) were female, and 116 residents (59%) were White. Mean Step 1 and Step 2 CK scores were 245.5 (SD 12.0) and 258 (SD 10.8) respectively. Step 1 scores, Alpha Omega Alpha membership, medicine clerkship grades, and interview scores were not associated with Milestone performance in the bivariate analysis and were not included in the multivariable model. In the multivariable model, overall clerkship grades, ranking of the medical school, and year entering residency were significantly associated with Milestone performance (P ≤ .04). Most traditional metrics used in residency selection were not associated with early performance on ACGME Milestones during internal medicine residency.

Female Residents Give Themselves Lower Scores Than Male Colleagues and Faculty Evaluators on ACGME Milestones

Orthopedic surgery is one of the specialties with the lowest number of women residents and practicing surgeons. The gender discrepancy in orthopedic residency training may drive a competency bias. We asked whether female orthopedic surgery residents score themselves lower on the Accreditation Council for Graduate Medical Education (ACGME) Milestones than their male counterparts, and lower than their faculty evaluators. We conducted a retrospective review of ACGME Milestone data from faculty and residents over a 4-year period. The data were analyzed using a snapshot of PGY2 (n = 20 residents) and PGY4 (n = 19 residents) scores, and using a Generalized Estimation Equation (GEE) to account for additional data points from the same residents over the 4-year data collection period. Assessment scores were compiled from a single orthopedic surgery residency at Oregon Health & Science University from 2014 to 2017. The residency program has 5 residents in each program year (PGY1 through PGY5); a total of 25 residents during each year of the study were included. On average, female residents scored themselves lower than both their male counterparts and their faculty mentors. Female PGY2 self-evaluation scores were lower than males in both patient care (p = 0.005) and medical knowledge (p < 0.001). When the GEE model was applied to 99 responses from 41 residents over a 4-year period, there were no gender-related differences in resident self-evaluation scores and in faculty scores of male and female residents, with the exception of meniscal tear. For this milestone, faculty rated female residents lower than males. Furthermore, the differences between faculty evaluation scores and resident self-evaluation scores were significantly lower for males than for females for 4 of the clinical domains, as well as the systems-based practice domains of cost and communication. Our results indicate female residents are at risk for a competency bias during training, as reflected by evaluations using the ACGME Milestones.

Social Media Bridges the Training Gap Between Match Day and Internship With ACGME Milestone-based Clinical Case Curriculum.

The objective was to bridge the relative educational gap for newly matched emergency medicine preinterns between Match Day and the start of internship by implementing an Accreditation Council for Graduate Medical Education Milestone (ACGME)-based virtual case curriculum over the social media platform Slack. We designed a Milestone-based curriculum of 10 emergency department clinical cases and used Slack to implement it. An instructor was appointed for each participating institution to lead the discussion and encourage collaboration among preinterns. Pre- and postcurriculum surveys utilized 20 statements adapted from the eight applicable Milestones to measure the evolution of preintern self-reported perceived preparedness (PP) as well as actual clinical knowledge (CK) performance on a case-based examination. A total of 11 institutions collaborated and 151 preinterns were contacted, 127 of whom participated. After participating in the Slack intern curriculum (SIC), preinterns reported significant improvements in PP regarding multiple Milestone topics. They also showed improved CK regarding the airway management Milestone based on examination performance. Implementation of our SIC may ease the difficult transition between medical school and internship for emergency medicine preinterns. Residency leadership and medical school faculty will benefit from knowledge of preintern PP, specifically of their perceived strengths and weaknesses, because this information can guide curricular focus at the end of medical school and beginning of internship. Limitations of this study include variable participation and a high attrition rate. Further studies will address the utility of such a virtual curriculum for preinterns and for rotating medical students who have been displaced from clinical rotations during the novel coronavirus pandemic.

Application Factors May Not Be Predictors of Success Among General Surgery Residents as Measured by ACGME Milestones

BackgroundCan factors within the Electronic Residency Application Service application be used to predict the success of general surgery residents as measured by the Accreditation Council for Graduate Medical Education (ACGME) general surgery milestones? MethodsThis is a retrospective study of 21 residents who completed training at a single general surgery residency program. Electronic Residency Application Service applications were reviewed for objective data, such as age, US Medical Licensing Examination scores, and authorship of academic publications as well as for letters of recommendation, which were scored using a standardized grading system. These factors were correlated to resident success as measured by ACGME general surgery milestone outcomes using univariate and multivariate analyses. This study was conducted at a single academic tertiary care and level 1 trauma facility. Residents who completed general surgery residency training from the years of 2012-2018 were included in the study. ResultsThere were few correlations between application factors and resident success determined by the ACGME milestones. ConclusionsApplication factors alone do not account for ongoing growth and development throughout residency. Unlike the results presented in the literature for other surgical subspecialties, predicting general surgery resident success based on application factors is not straightforward.

A National Survey of Integrated Vascular Surgery Residents' Experiences With and Attitudes About Quality Improvement During Residency

Integrated vascular surgery residency, or "0+5," programs provide education in the Accreditation Council for Graduate Medical Education (ACGME) competencies of Systems-Based Practice (SBP) and Practice-Based Learning and Improvement (PBLI), which include milestones related to quality improvement (QI). It is unclear what QI curricula are in place in 0+5 programs nationally or how 0+5 residents perceive the importance of QI. The purpose of this study is to assess current 0+5 residents' knowledge, experiences with, and attitudes about QI. A survey was developed using the ACGME Common Program Requirements and Milestones pertaining to QI. All 0+5 residents from 2017 to 2018 academic year were emailed an electronic link to the survey. Descriptive statistics and cross-tabulations were calculated using Stata/MP version 13.1. All 0+5 vascular surgery residency programs in the United State (n = 52). The survey was completed by 35% (n = 90/257) of 0+5 residents, representing 75% of 0+5 programs in the United States (n = 39/52). Forty-one percent of respondents felt that applying QI methods is very important and 33% felt that QI education is very important for their future work, however, just 13% felt very prepared to lead a QI initiative. Residents' perceptions of preparedness to lead QI projects and the importance they attached to QI education were significantly influenced by their participation in a QI project (p = 0.003 and p = 0.038 respectively). Finally, just 8% (n = 6) of residents responded correctly to all 13 knowledge-based questions and these residents felt better prepared to lead a QI initiative compared to those who answered incorrectly (p = 0.002). Most 0+5 residents report participation in a QI project during residency, however, few feel prepared to lead a QI initiative in practice. Furthermore, only half of PGY5 0+5 residents report achieving specific ACGME targets for graduation pertaining to QI. Current QI curricula in 0+5 programs may be inadequate in teaching fundamental QI concepts and achieving ACGME competency targets for graduation.

A Qualitative Study of Independent Home Visits by Hospice Fellows: Addressing Gaps in ACGME Milestones by Fostering Reflection and Self-Assessment.

With the rapid growth in the number of fellowship programs in Hospice and Palliative Medicine (HPM), many are in the process of developing ways to demonstrate that fellows are attaining educational milestones. Reflection and self-assessment are key components of 2 Accreditation Council for Graduate Medical Education (ACGME) competencies, practice-based learning and improvement, and systems-based practice, which have both been historically challenging to learn and assess. This article describes results of a content analysis of narrative data collected from HPM fellows' self-assessments as they performed hospice home visits independently in a new clinical rotation. This was a prospective qualitative study. Eight fellows completed 217 unsupervised hospice home visits from 2014 to 2016. Fellows completed weekly self-assessment forms, which captured both clinical visit information and practice data elicited from responses to open-ended reflection prompts. Analysis of 29 self-assessment forms generated 6 themes: patient- and family-centered care, self-efficacy, systems-based care, commitment to doing their best, catalyst for professional growth, and purpose and meaning in work. The fellows recognized numerous barriers distinct to providing care in homes. All fellows felt prepared to perform home visits throughout the rotation and after training. Narrative data collected during the independent home visit rotation provided evidence that HPM fellows detected gaps in their performance, planned for practice improvements in subsequent visits, and valued working within an interprofessional team. Built-in opportunities for fellows to reflect during training are critical in meeting ACGME milestones, and are integral to their professional development.

A Content Analysis of the ACGME Specialty Milestones to Identify Performance Indicators Pertaining to the Development of Residents as Educators

Construct: For curriculum development purposes, this study examined how the development of residents as educators is reflected in the Accreditation Council for Graduate Medical Education (ACGME) Milestones. Background: Residents teach patients, families, medical students, physicians, and other health professionals during and beyond their training. Despite this expectation, it is unclear how the development of residents as educators is reflected in the specialty-specific Milestones. Approach: We performed a textual content analysis of 25 specialty Milestone documents available as downloads from the ACGME website in December 2015. Syntactical units of interest included developmental progressions that describe the development of educators over the course of residency training and 16 key terms identified during the analysis. We then categorized the terms by associated Milestone level, ACGME core competency, and targeted learner(s). Results: We identified 10 developmental progressions and 546 instances of the 16 key terms that describe the development of physician educators. The frequency of terms among specialties was quite variable (5–46 terms per specialty, Mdn = 21). The majority of education-related terms appeared at advanced Milestone levels; there were 139 (26%) such instances in Level 4 and 296 (54%) in Level 5. Education-related terms were identified in all six ACGME core competencies, with greatest frequency in Patient Care (157, 29%). Other residents were the learners most frequently targeted by education-related Milestones (211, 40%). Conclusions: The current ACGME Milestones largely imply that resident teaching is a high-level or aspirational goal, achieved without a clear or consistently assessed developmental progression. These findings run counter to the theoretical basis that underlies the development of the Milestones. Wide variation among specialties indicates lack of consensus around the ideal skill set of the resident educator and limits the utility of these documents for curriculum development in this domain.

Reader response: Education research: The current state of neurophysiology education in selected neurology residency programs.

I read with interest the Resident & Fellow Education Research article by Daniello and Weber1 on the current state of neurophysiology education in neurology residency programs. The authors found that the majority of residents graduate without meeting level 4 Accreditation Council for Graduate Medical Education (ACGME) milestones in EEG and EMG (EMG > EEG). Inpatient service demands and time constraints on residents were reported as the major cause of resident failure to meet ACGME milestones. The last few decades have witnessed a knowledge explosion in neurology. Somehow, the knowledge explosion and knowledge divide needs to be bridged, especially by residents in training. The 80-hour work week restriction placed by ACGME needs to be relaxed so that residents can be exposed to various neurology subspecialties, or we risk having many of our patients with epilepsy and neuromuscular disorders receive suboptimal care from these doctors in the future.

119 Development of a Clinical Performance Dashboard to Empower Resident Education

To develop an emergency medicine (EM) resident clinical dashboard to address the need for more objective and formative feedback in medical education. This dashboard contains high-impact visual displays of select performance measures for EM residents based on data extracted from the electronic medical record (EMR). This was a feasibility study conducted at a four-year EM residency program at an academic tertiary care center. Resident performance data from July-Nov 2016 were extracted from the EMR. Residents’ names were replaced with a random two-letter code (“AA,” “DE,” etc) to protect resident confidentiality. A business analytics software was used to calculate 4 performance measures, or key performance indicators (KPIs). KPIs were designed to 1) reflect state or nationally reported quality metrics; 2) represent metrics supported by the American College of Emergency Physicians (ACEP) Clinical Emergency Data Registry (CEDR); or 3) represent Accreditation Council for Graduate Medical Education (ACGME) EM Milestones. These KPI were coded into an interactive interface to create the EM resident clinical Dashboard. Data filters were added to allow visualization based on resident training level, patient acuity, and ED location. Descriptive statistics were calculated for each KPI. We developed a prototype interactive clinical dashboard measuring the following KPIs: admission rate, time to disposition and time to discharge. Interactive graphs were generated for each KPI, and residents were ranked and color-coded by tertile for easy visual interpretation (see accompanying figure for example). Different colors were assigned to the top-performing tertile (lowest admission rates, shortest disposition and discharge times, and highest sepsis protocol compliance), the middle tertile, and the bottom tertile. For all patients, the average admission rate was 20% for PGY1s, 23% for PGY2s, 29% for PGY3s, and 23% for PGY4s. average time to disposition in minutes was 338 for PGY1s, 218 for PGY2s, 226 for PGY3s, and 216 for PGY4s. Average Time to Discharge in minutes was 436 for PGY1s, 305 for PGY2s, 310 for PGY3s, and 272 for PGY4s. In this feasibility study, we demonstrated that an EM Clinical resident dashboard can be generated from EMR data to display and manipulate resident performance measures across multiple KPIs. We hope that such a Dashboard can be further developed as an innovative tool to enhance and supplement usual feedback in order to improve resident learning and career preparedness. Future research plans include a pre-post analysis of resident behavior after incorporation of the dashboard into ACGME-mandated performance review sessions, the inclusion of additional KPIs such as opioid prescribing practices and CT utilization, and using objective dashboard data to quantify measures of competency within the ACGME Milestones.