The Emergency Medicine Milestones: With Experience Comes Suggestions to Improve.

Abstract

In the November issue, an article, “Goal-directed Focused Ultrasound Milestones Revised: A Multiorganizational Consensus,”1 recommends significant change to the patient care (PC)12 subcompetency of the emergency medicine (EM) Milestones. The membership of this consensus group spans multiple organizations and individuals interested in emergency ultrasound (EUS) as well as resident education. This effort to improve this particular subcompetency is to be lauded. An iterative improvement to the progression of Milestones within PC12 can be justified on the basis of a large body of work produced after the introduction of the EM Milestones. However, there are a number of misperceptions within the article that deserve discussion. As a member of the original Emergency Medicine Milestone Working Group (EMMWG) I can address these suggested changes. The EM Milestones were introduced formally to the EM community in 2012 and published as an article in 2013.2 The EM Milestones are extensions, or subcompetencies, of the six original general competencies introduced in 1999 by the Accreditation Council for Graduate Medical Education (ACGME).3 Milestones are markers of specialty-specific achievements that occur as residents progress through training.4 The goal of the original EMMWG, consisting of 10 individuals representing the diverse organizations of EM, was to develop a consensus set of Milestones unique to EM within each of the six general competencies. This history is particularly important to PC12 because although the EMMWG members had some knowledge of EUS and EUS education, none had a background in EUS that matches the extent of knowledge and training of the consensus group members recommending changes to the PC12 subcompetency. Informal consultation occurred, but because of time constraints in the development of the entire set of EM Milestones a lengthier task force process for individual subcompetencies could not be considered. The development of the EM Milestones consisted of three components—the PC subcompetencies; the procedural subcompetencies within the patient care general competency; and the general competencies of practice-based learning and improvement (PBLI), interpersonal and communication skills (ICS), systems-based practice (SBP), and professionalism (P). The EMMWG was guided in the development of the patient care subcompetencies by the Model of the Clinical Practice of Emergency Medicine (Model). Although multiple iterations have occurred, Hockberger and his group5 introduced a three-dimensional matrix in the 2001 version consisting of conditions and components, patient acuity, and most importantly for EM Milestones development, physician tasks. The American Board of Emergency Medicine (ABEM) took these physician tasks and incorporated them into knowledge, skills, and abilities (KSAs) within an internal initial certification task force (ICTF). This task force benefitted from previous Model studies validating the Model.6 The KSAs developed by the ICTF guided the EMMWG in their development of the PC subcompetencies, including incorporation of the ABEM-defined certification-level KSAs into Level 4 of the PC subcompetencies. The general competencies of PBLI, ICS, SBP, and P had had multidisciplinary expert panels supported by the ACGME develop generic Milestones for each. The EMMWG took this work and modified each of the generic general competencies to be more specific to the needs of EM. Finally, the procedural PC subcompetencies were developed de novo with limited prior published recommendations or guidelines to go by. In the case of EUS the EMMWG had multiple concerns. One of the largest concerns was whether EUS, traditionally a radiologic modality, would be accepted by the ACGME as one of EM's Milestones subcompetencies. Prior work by the ACGME Review Committee in Emergency Medicine had integrated EUS into the program requirements for EM resident training. At the time of the development of the EM Milestones it was not known whether EUS would be allowed, although inclusion in the specific EM program requirements supported allowance. This led to discussion of how detailed to make PC12. If PC9 (general approach to procedures), PC10 (airway management), PC11 (anesthesia and acute pain management), PC13 (wound management), and PC14 (vascular access) are considered, they each have 11, 15, 11, 17, and 14 Milestones, respectively, compared to seven for PC12. Clearly these other procedural-based subcompetencies contain greater detail. With the increased acceptance and use of US by not just EM but of other specialties a better described progression of Milestones within PC12, including the use of EUS for EM procedures can now be justified. The level of detail contained in each subcompetency was purposely held to a level that was fairly general and broad. It was not the goal of the EMMWG to produce individual curricular documents for each subcompetency. Rather, the goal was to develop a set of progressive markers of achievement or proficiency for each of the subcompetencies. Likewise, it was not the goal of the EMMWG to develop assessment tools or for the Milestones to become the assessment tools of each subcompetency. The hope was that best practices would emerge that consisted of validated assessment tools that would inform the Clinical Competency Committee (CCC) of the proficiency level within each subcompetency for a given resident. Each of these Milestones in all subcompetencies went through a vetting process in which core faculty of residencies across the country assigned Milestones to specific proficiency levels in a blinded fashion.7 Each subcompetency, including PC12, had many of their individual Milestones rewritten or assigned to a different proficiency level compared to the initial assignment by the EMMWG consensus process. The resulting EM Milestones became a product of a thorough multistage validation process. Any significant revision of the EM Milestones runs the risk of arbitrary assignment to a specific proficiency level without the degree of validation of the original EMMWG's work. This article states that its goal is to “provide revised milestone levels for assessment of resident performance.” The Milestones serve as an assessment of a resident's progression within a subcompetency. Confusion occurs when the EM Milestones are considered the tool for resident assessment. The article asserts that, “Since the introduction of the EM milestones, many ultrasound and education leaders have not changed their educational strategy or their assessment of resident POC ultrasound education.” The authors argue that this is due to the inapplicability of portions of PC12. The EM Milestones when developed were not intended to alter educational strategy or assessment. Assessment tools in use would need to inform the CCC of proficiency placement, but would not necessarily need to be newly developed. Likewise, a program effectively teaching EUS would not need to alter its curricula. If other procedural subcompetencies are considered, such as PC10 (airway management), it would be misleading to state or believe that this developed subcompetency would cause residencies to alter their educational curricula or even their assessment methods. This article recommends 21 different Milestones across the five levels of proficiency, compared to the seven that currently exist. This is a level of granularity that the original EMMWG avoided to not become a substitute for a curriculum in EUS or be used as an assessment tool. In some cases the recommended Milestones for each level do not match up to current practice. In Level 1, expected of a graduated medical student, it is recommended that four Milestones be completed. These include describing relevant anatomy, the indications, hand motions with proper manipulation, as well as performance of an eFast. The indications for goal-directed clinical US as well as performance of an eFast were considered to be Level 2 in the national survey of core faculty. Additionally, the article cites as evidence an article that surveyed allopathic medical schools related to their undergraduate US education.8 This study had 82 of 134 schools responding, and the same article said that 19% said US was a priority. This survey did not include osteopathic medical schools, whose graduates make up 9% of the EM resident pool.9 This article noted only 60% of respondents stated US was integrated into their curriculum, and nearly 80% agreed that US should be part of medical education curriculum. US education in the medical school curriculum may not be as uniform as presented. It is not clear that graduating medical students would uniformly be able to demonstrate hand motions of scanning, perform an eFAST exam, or describe the difference of goal-directed clinical ultrasound and a consultative ultrasound. Strategically it may be valuable to define these Milestones to ensure that undergraduate education attains or provides this. However, it is not clear that the recommended Milestones reflect current US education. The article recommends four Milestones for Level 2. The first is similar to the current Level 2 Milestone. The other three define basic skills. When the next ACGME-ABMS sponsored revision of the Milestones occur, a question the EMMWG 2.0 will need to answer is whether to consider various categories of EUS progression—clinical goal-directed US (eFAST, DVT studies, etc.), EUS for procedural guidance (central lines, abscess drainage, etc.), and resuscitative (IVC measurements, pericardial, etc.). If these categories are included, there should be a progression of proficiency from Level 2 to at least Level 4 in each category. In Level 3, the fourth recommended Milestone moves performance of a minimum of 150 scans from Level 4 to Level 3. This recommended Milestone expands the Milestone to state, “with scans in each core application,” although it is unclear what the core applications are. The article argues that a stipulated number of scans fails to account for clinical integration. I would argue that moving the 150 scans to Level 3 is impractical in that residents will need almost 3 years to achieve this in many programs. The experience gained with each EUS is iterative and would coincide with the recommended Level 4 Milestone of “Consistently utilizes and integrates appropriate ultrasound applications in clinical management.” When PC14 (vascular access) and PC10 (airway management) are reviewed, a specific minimum number of procedures is recommended within Level 4. Whether 150 scans is the right number can be debated, but a minimum of 150 total EUS examinations is recommended by the American College of Emergency Physicians.10 The last Milestone in Level 3 is “Uses ultrasound for dynamic guidance of procedures.” The previous comment about expansion of EUS Milestones to describe advancement within different categories of EUS use applies here. In the recommended Level 4 Milestones, the first describes competency in documentation of clinical US in the medical record. This is a redundant Milestone and is covered in technology (SBP3) as a Level 2 Milestone, “Ensures that medical records are complete, with attention to preventing confusion and error.” Although correct documentation of an EUS is desired, it is no less desired for airway management, vascular access, etc. By putting this Milestone within SBP3 a general documentation statement could be made, applicable to all procedures and care in the ED. Level 5 Milestones are generally thought of as being attainable after significant experience. The implication is not that experience should cause all EM physicians to attain Level 5. Level 5 is aspirational and depending on the subcompetency may involve more advanced applications, policy development, quality reviews, or teaching. The recommended Level 5 Milestones touch upon these areas. The authors of the article “Goal-directed Focused Ultrasound Milestones Revised: A Multiorganizational Consensus,”1 should be congratulated for seeing a need for an iteration of the PC12 subcompetency and organizing EUS educators to address this need. This article brings into focus the intent of the Milestones as markers of proficiency progression rather than as a curriculum document or assessment tool. It is exactly what was hoped for—increased dialog related to resident assessment as well as the emergence of best practices to evaluate each of these subcompetencies. This article will benefit the next EMMWG as it considers changes to various subcompetencies.