In a recent issue, Norman et al1 and Carter2 use evocative metaphors to describe the status of the Milestone project. Norman et al1 summarize the promise of competency-based education/Milestones and categorically state, “Regrettably, these declarations appear to be more a matter of faith than of evidence.”1 Although, at this point, there is insufficient evidence that this major overhaul will be meaningful, its transformational potential remains promising. In support of the Milestones, Carter2 states that “the Next Accreditation System (NAS) and Milestones are akin to designing a plane in flight. . . . Starting in July 2014, we will all be aboard this plane that is now in midflight.” Carter's statement emphasizes an important point: Milestones may be a very good idea, but their greatest challenge lies in it being an incomplete and unproven hypothesis. The next step is to replace faith with evidence. The traditional scientific approach to developing an untested hypothesis (the Milestones) into a proven theory would require studies to answer the essential question as to whether their adoption will produce better doctors in unsupervised practice and to have as many of these (positive) answers before the Milestones are broadly prescribed. For instance, it can be hypothesized that the elaborate Milestones are better than the current system of 6 general competencies, which it seeks to replace. However, the previous hypothesis—that adoption of the 6 general competencies would lead to better physicians—was never tested, and it will be a while before there are data to support the hypothesis that Milestones will be an improvement over the general competencies. This is an opportunity to sift through a bank of competencies, generate data, and then pick only those competencies that are proven to produce good physicians. For instance, do we need more (eg, interprofessional collaboration, personal and professional development),3 fewer, or different competencies and Milestones? There are suggestions that Milestones will be more expensive, at least to the consumers, so we need evidence that this extra expense is worthwhile. It is important to define outcomes a priori: better at what? Milestones carry the promise to be useful in giving more detailed and accurate feedback to trainees but do not address the critical question: Will their use make better physicians? Will there be better clinical outcomes4 or fewer adverse events5 or more cost-effective care? A positive answer to these questions is needed to make these changes meaningful and longitudinally successful. It is heartening that there are ongoing studies of the Milestones in several disciplines. These are difficult to conduct and even more difficult to fund. Therefore, we applaud the Accreditation Council for Graduate Medical Education, which “although not a funding agency, has agreed to use interest from reserves (but no current accreditation fee revenue) to provide seed funding”6 for independent educational research on other important questions. This may infuse foundations and government agencies with more confidence to support studies of critical projects, such as the Milestones. We are clearly on the right track. Addressing this promising hypothesis with scientific rigor and prudence would lead us to delay full implementation of the unproven Milestones; to conduct studies to refine their specifics, prove clinical relevance, and minimize the burden; and then to confidently prescribe a proven remedy that reliably leads to graduating the finest physicians.
Read full abstract