Marking the path forward with milestones.

Abstract

In this issue of Pediatrics Schumacher et al1 provide an analysis of the resource domains needed to implement the Pediatrics Milestones into residency training. While noting the promise of Pediatrics Milestones “for developing a valid . . . way to track residents’ development and provide a roadmap for their learning,” the authors question the ability of training programs to marshal necessary resources. Faculty raise similar concerns with variants of the following questions: Why change (from the current training and assessment system)? What is the evidence (that milestone assessment works)? and What can be done to increase efficiencies and limit unnecessary costs?Even if there were no regulatory mandate for milestones from the Accreditation Council for Graduate Medical Education, adhering to the old apprentice training and assessment model has been superseded by a host of external forces including societal insistence on patient safety and improved health outcomes.2 A learner-centered approach to training and assessment (competency-based medical education [CBME]), predicated on the needs of the populations served, has been the response to these forces. This approach allows faculty and residents to jointly assess the resident’s progress toward specific behavioral milestones on the road toward mastery.3,4Milestone implementation will require incremental resources, but the current system also has costs, albeit less visible ones borne by trainees who receive inaccurate feedback5 and patients who experience substandard care associated with deficits in residency training.6 The ultimate perspective on costs is a societal one, where the current investment in medical education of a mere 0.3% of total health expenditures may be insufficient to produce desired outcomes.7“Where is the evidence that CBME produces better doctors?” There are 2 major barriers to answering this question: a limited number of assessment tools with the validity evidence to make accurate judgments about trainee performance and learner outcomes and an inability to link educational outcomes to patient care outcomes. But research in this area is already underway. A small pediatrics pilot study has demonstrated the feasibility and utility of strategies and tools to assess milestones.8,9 Larger-scale studies of competencies and milestones are in the planning phase and will begin from a position of equipoise and a commitment to methodological rigor to develop tools with the validity evidence to determine whether milestones improve educational outcomes for learners. Data Commons (http://mydatacommons.org/), a consortium of some major medical education and certifying organizations, is creating an information technology hub that could link deidentified educational and patient care outcome data.Designing assessment tools and strategies across the educational continuum allows for efficiencies of scale. Medical school faculty can focus on the same core competencies for students and residents by using assessment tools that are generalizable to both. Similarly, trainees who have been exposed to real-time competency assessments in medical school should adapt readily to CBME in residency.Faculty development can be targeted to the needs of the faculty. Although all teaching faculty need basic direct observation skills, a small cadre will need the expertise to lead others in assessment. The latter group needs financial support and academic recognition independent of the Milestone Project. Medical education has become a subspecialty that requires a body of knowledge and skill beyond the generalist. Acknowledgment of this expertise is overdue.The data management systems and assessment tools, particularly those providing information needed for high-stakes decisions, should be developed as part of a collaborative effort, thus sparing each individual program from having to repeat what others have done. The American Board of Pediatrics will work with others to advance that objective. We support the other proposed solutions,1 namely handheld devices at the point of care, relational database support of a residency management system that synthesizes information from an array of tools, and the ability of this system to generate formative reports for learners as well as evidence of learner and program progress for institutions and regulatory bodies.We also agree with Schumacher et al1 that all regulatory bodies involved in resident education should collaborate to help programs avoid duplication of effort, make data submission easier, and satisfy multiple regulatory requirements with the single submission of 1 data set. Isolated discussions have begun between regulatory bodies, but it is time for a full-scale summit to target a systematic approach toward reduction of administrative burdens for residency training programs.Schumacher et al1 have advanced resident education by articulating the extensive resources necessary to fully implement the Pediatrics Milestones. The old assessment system is inadequate. The new one is untested. But, as in the past, a concerted effort by all invested in training future physicians can overcome the challenges for the benefit of trainees and their future patients.10