Milestone Project Research Articles

TF-5 Off-Service Resident ToolBox: A “Moment of Need” Emergency Medicine Curriculum for the Off-Service Rotator

Bedside and experiential learning are essential parts of emergency medicine education. Off-service residents (hereafter referred to as rotators) pose a particularly challenging group with which to engage in on-shift teaching due to their widely varying levels of prior knowledge and clinical experience. The Five Moments of Need Model described by Mosher and Gottfredson was designed for the corporate sector but offers significant parallels to the on-shift learning environment. E-Learning modalities lend themselves well meeting the resident in many “Moments of Need” and offer the rotator an opportunity to maximize their experiential learning during patient care. Prior efforts to develop a standard rotator curriculum predate the American College of Graduate Medical Education Milestones Project and have not been widely adopted. This curriculum uses an E-Learning repository as the core of a rotator curriculum. It augments a multi-modal approach addressing specialty-specific milestones for rotators. Study Objectives: To develop a rotator curriculum offering learning opportunities appropriate to several “moments of need.” After completion of this curriculum, rotators will: 1) Recognize patients with an unstable medical condition requiring acute stabilization, 2) Perform a focused history and physical exam appropriate to a patient’s chief complaint 3) Develop a differential diagnosis and treatment plan emphasizing acute life-threatening conditions. 4) Safely perform select emergency procedures under supervision. Rotators will be provided access to the Off-Service Resident ToolBox 2 weeks prior to the beginning of their rotation. The ToolBox is an online repository of brief written, video and PowerPoint-based modules. ToolBox content includes a practical introduction to the ED, a series of chief-complaint-focused clinical assessment modules, procedure videos, journal club materials, suggested reading and self-study quizzes. Rotators will be encouraged to review the materials prior to their rotation addressing the “Learning for the first time” and “Learning more” moments of need as well as on shift as preparation for patient evaluation addressing the “Applying what you’ve learned” and “When things change” moments of need. Rotators will apply knowledge gained through ToolBox materials and on-shift experience to their critical analysis of a variety of high-yield journal articles in a rotator-specific journal club led by emergency medicine faculty. Rotators will have 2 primary means of evaluation. Content-specific self-study modules are provided in the ToolBox to provide means of self-assessment. Rotators are evaluated by emergency medicine faculty through means of direct observation. Evaluations are milestone-based and completed via electronic survey response. Curriculum evaluation will be based upon before and after surveys assessing rotator confidence in clinical scenario management, content utilization, and satisfaction. Curricula designed for the rotating resident in the Milestones era are not widely available. The ToolBox offers an e-learning based rotator curriculum emphasizing provision of user-friendly modules at high-yield “moments of need” to maximize both clinical performance and experiential learning through direct patient care.

TF-12 Teaching Residents About Continuous Quality Improvement

Almost 2 decades ago, the Institute of Medicine released the report “To Err is Human” stating that at least 44,000 people died from preventable medical errors in hospitals, equating to between $17 billion and $29 billion in estimated financial costs. This provocative report called on the health care industry to create systems to address issues of patient safety. In training our current generation of physicians, it is important to not only train them to be remarkable clinicians, but also to be able to address issues of patient quality and safety. In line with this, emergency medicine residents ought to participate “in performance improvement to optimize patient safety” and achieve the graduation target of participating “in an institutional process improvement plan to optimize [emergency department] practice and patient safety” as stated in the Emergency Medicine Milestone Project. The primary goal is to equip residents with the tools necessary to engage in continuous quality improvement (CQI) as a way to address patient quality and safety initiatives. The objectives to reach this goal include: (1) Define basic terminology used in the patient safety and quality environment; (2) Explain the Model for Improvement and the steps in the Plan-Do-Study-Act (PDSA) cycle; (3) Create Aim-Measure-Change statements for a patient quality or safety case. The online module consists of the following sections: (1) Pre-test as outlined in the evaluation plan; (2) Why care about quality improvement; (3) What is CQI; (4) Model for Improvement; (5) Fishbone diagram for identifying cause and effect; (6) PDSA cycle; (7) Interpreting data with a run chart; (8) Post-test as outlined in the evaluation plan. Much of the material is curated from the Institute for Healthcare Improvement Open School. Both the pre-test and post-test will include 3 parts: (1) Survey with questions on the learner’s comfort level with approaching and starting a CQI project; (2) Quiz on basic terms used in patient quality and safety; (3) Case-based quiz asking learners to write statements addressing the aim, identifying a measure, and creating a recommendation of a change for the presented case, which will be graded using the Quality Improvement Knowledge Application Tool Revised scoring system. This asynchronous training module is an interactive computer-aided instruction created to help residents achieve a foundation of knowledge to be able to develop and implement a CQI project.

Use of a Clinical Pathologic Conference to Demonstrate Residents' ACGME Emergency Medicine Milestones, Aid in Faculty Development, and Increase Academic Output.

The Emergency Medicine Milestones Project, developed by the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Emergency Medicine, includes competence targets for residents to attain and, ultimately, to exceed American Osteopathic Association and ACGME expectations for residents. The authors sought to use the clinical pathologic conference (CPC) format in their institutions' Emergency Medicine Milestones Project to provide measurable residency academic and faculty development outcomes. The CPC is an event in which a resident presents an unknown case to a discussant in advance of a didactic session to demonstrate an organized approach and decision-making rationale to a differential diagnosis. Feedback forms included the assessment of resident discussants from the perspective of level-5 Milestone achievements in particular. Developing an internal CPC competition with a dedicated core faculty coordinator who provides skill development for both resident and faculty presentation has proven successful. Such a competition can document the level-5 achievements for senior residents, be a source of faculty development, and increase peer-reviewed academic output.

Resident-Specific Morbidity Reduced Following ACS NSQIP Data-Driven Quality Program

The Accreditation Council for Graduate Medical Education Milestone Project for general surgery provided a more robust method for developing and tracking residents' competence. This framework enhanced systematic and progressive development of residents' competencies in surgical quality improvement. A 22-month interactive, educational program based on resident-specific surgical outcomes data culminated in a quality improvement project for postgraduate year 4 surgery residents. Self- assessment, quality knowledge test, and resident-specific American College of Surgeons National Surgical Quality Improvement Program Quality In-Training Initiative morbidity were compared before and after the intervention. Quality in-training initiative morbidity decreased from 25% (82/325) to 18% (93/517), p = 0.015 despite residents performing more complex cases. All participants achieved level 4 competency (4/4) within the general surgery milestones improvement of care, practice-based learning and improvement competency. Institutional American College of Surgeons National Surgical Quality Improvement Program general surgery morbidity improved from the ninth to the sixth decile. Quality assessment and improvement self-assessment postintervention scores (M = 23.80, SD = 4.97) were not significantly higher than preintervention scores (M = 19.20, SD = 5.26), p = 0.061. Quality Improvement Knowledge Application Tool postintervention test scores (M = 17.4, SD = 4.88), were not significantly higher than pretest scores (M = 13.2, SD = 1.92), p = 0.12. Sharing validated resident-specific clinical data with participants was associated with improved surgical outcomes. Participating fourth year surgical residents achieved the highest score, a level 4, in the practice based learning and improvement competency of the improvement of care practice domain and observed significantly reduced surgical morbidity for cases in which they participated.

Overview on the studies of overseas physician′s competency assessment

This research introduced four models of overseas physician competency assessment, and their methodology, and hotspot Milestone project in the USA. The authors propose to build a national assessment benchmark framework in favor of industry consensus. In terms of physician assessment methods, they recommended to test the validity and reliability of these existing tools, rather than merely focusing on the development of new assessment tools. Regarding physician assessment phasing, the paper proposed to build a dynamic and process-based assessment system. Key words: Health manpower; Competency; Assessment; Model; Development

Towards Consensus: Training in Procedural Skills for Diagnostic Radiology Residents—Current Opinions of Residents and Faculty at a Large Academic Center

The Diagnostic Radiology Milestones Project provides a framework for measuring resident competence in radiologic procedures, but there are limited data available to assist in developing these guidelines. We performed a survey of current radiology residents and faculty at our institution as a first step toward obtaining data for this purpose. The survey addressed attitudes toward procedural standardization and procedures that trainees should be competent by the end of residency. Current residents and faculty members were surveyed about whether or not there should be standardization of procedural training, in which procedures residents should achieve competency, and the number of times a procedure needs to be performed to achieve competency. Survey data were received from 60 study participants with an overall response rate of 32%. Sixty-five percent of respondents thought that procedural training should be standardized. Standardization of procedural training would include both the list of procedures that trainees should be competent in at the end of residency and the standard minimum number of procedures to achieve competency. Procedures that both residents and faculty agreed are important in which to achieve competency included central line/port procedures; CT-guided abdominal, thoracic, and musculoskeletal procedures; minor fluoroscopic-guided procedures; general fluoroscopy; peripheral line placements; and US-guided abdominal procedures. For most of these categories, most respondents believed that these procedures needed to be performed 6-20 times to achieve competency. Both resident and faculty respondents agreed that procedural training should be standardized during residency, and competence in specific procedures should be achieved at the completion of residency. Although this study is limited to a single institution, our data may provide assistance in developing future guidelines for standardizing image-guided procedure training. Future studies could be expanded to create a national consensus regarding the implementation of the Diagnostic Radiology Milestones Project.

Neurosurgical Resident Error: A Survey of U.S. Neurosurgery Residency Training Program Directors' Perceptions.

Efforts to address resident errors and to enhance patient safety have included systemic reforms, such as the Accreditation Council for Graduate Medical Education's (ACGME's) mandated duty-hour restrictions, and specialty-specific initiatives such as the neurosurgery Milestone Project. However, there is currently little data describing the basis for these errors or outlining trends in neurosurgical resident error. An online questionnaire was distributed to program directors of 108 U.S. neurosurgery residency training programs to assess the frequency, most common forms and causes of resident error, the resulting patient outcomes, and the steps taken by residency programs to address these errors. Thirty-one (28.7%) responses were received. Procedural/surgical error was the most commonly observed type of error. Transient injury and no injury to the patient were perceived to be the 2 most frequent outcomes. Inexperience or resident mistake despite adequate training were cited as the most common causes of error. Twenty-three (74.2%) respondents stated that a lower post graduate year level correlated with an increased incidence of errors. There was a trend toward an association between an increased number of residents within a program and the number of errors attributable to a lack of supervision (r=0.36; P= 0.06). Most (93.5%) program directors do not believe that mandated duty-hour restrictions reduce error frequency. Program directors believe that procedural error is the most commonly observed form of error, with post graduate year level believed to be an important predictor of error frequency. The perceived utility of systemic reforms that aim to reduce the incidence of resident error remains unclear.

Using the Teaching Perspectives Inventory as an Introduction to a Residents-as-Teachers Curriculum

The Anesthesiology Milestone Project includes a milestone for assessing the teaching attributes of residents within the competency of Practice-based Learning and Improvement. We intend to develop a Residents-as-Teachers educational curriculum to assist our residents in successfully achieving this milestone. The goal of this study is to identify the specific teaching perspectives and intentions of our residents and to promote residents' comprehension of their own teaching philosophy. We invited our residents to complete the Teaching Perspective Inventory (TPI) and a follow-up survey to gather information regarding dominant and recessive teaching perspectives, their intended career pathway, and their view of the importance of understanding teaching perspectives. The two most common dominant teaching perspectives are apprenticeship and nurturing for residents who are planning a career in both academic medicine and private practice. A greater percentage of residents planning an academic career agree that identifying their teaching perspective is beneficial to their role as a clinical educator, compared to those anticipating a career in private practice. Based on this pilot data, our Residents-as-Teachers curriculum will include instruction of educational strategies specifically designed towards the apprenticeship and nurturing perspectives.

Ways to Write a Milestone: Approaches to Operationalizing the Development of Competence in Graduate Medical Education

To identify approaches to operationalizing the development of competence in Accreditation Council for Graduate Medical Education (ACGME) milestones. The authors reviewed all 25 "Milestone Project" documents available on the ACGME Web site on September 11, 2013, using an iterative process to identify approaches to operationalizing the development of competence in the milestones associated with each of 601 subcompetencies. Fifteen approaches were identified. Ten focused on attributes and activities of the learner, such as their ability to perform different, increasingly difficult tasks (304/601; 51%), perform a task better and faster (171/601; 45%), or perform a task more consistently (123/601; 20%). Two approaches focused on context, inferring competence from performing a task in increasingly difficult situations (236/601; 29%) or an expanding scope of engagement (169/601; 28%). Two used socially defined indicators of competence such as progression from "learning" to "teaching," "leading," or "role modeling" (271/601; 45%). One approach focused on the supervisor's role, inferring competence from a decreasing need for supervision or assistance (151/601; 25%). Multiple approaches were often combined within a single set of milestones (mean 3.9, SD 1.6). Initial ACGME milestones operationalize the development of competence in many ways. These findings offer insights into how physicians understand and assess the developmental progression of competence and an opportunity to consider how different approaches may affect the validity of milestone-based assessments. The results of this analysis can inform the work of educators developing or revising milestones, interpreting milestone data, or creating assessment tools to inform milestone-based performance measures.

Educational Resources for Resident Training in Quality Improvement: A National Survey of Urology Residency Program Directors

IntroductionA key physician competency outlined in the Urology Milestone Project is engagement in quality improvement. Despite this mandate little is known about the attitudes of urology residency program directors regarding the relative importance of quality improvement education. Therefore, we performed a national survey of program directors. MethodsA 25-item survey was developed to investigate program director knowledge and training in quality improvement methodology, participation in quality improvement related activities, curriculum support for resident quality improvement educational activities, and attitudes regarding the relative importance of quality improvement education. The survey was sent via e-mail (November 1, 2016) to all program directors affiliated with the Society of Academic Urologists (sample size 116 of 134, 87% of Accreditation Council for Graduate Medical Education programs). ResultsA total of 36 program directors returned a completed survey for a response rate of 31%. Only 22% (8) of program directors reported receiving formal education or training in quality improvement methodology. Overall 44% (16) of program directors reported that their program offers formal education or a curriculum in quality improvement methodology for their trainees. Program directors expressed a strong desire for residents to learn quality improvement methodology (positive response 32 of 36, 89%) and understand how to apply it to conduct a quality improvement project (positive response 30 of 35, 86%). Program directors strongly believe that a urology oriented quality improvement curriculum would be a valuable resource (positive response 31 of 36, 86%) with a need for support from our professional society (positive response 29 of 36, 81%). ConclusionsA minority of programs have quality improvement education available for residents. However, program directors agree that quality improvement is an integral part of residency training that should be promoted by our profession.

Clinical Pharmacist Faculty Contribution to Evidence-Based Medicine and Milestones.

Family medicine residency programs (FMRPs) endeavor to meet evidence-based medicine (EBM) subcompetencies through the milestones project. Comprehensive descriptions of clinical pharmacists' contributions in teaching EBM within the context of residency are limited. Over a study period of 2 months, clinical pharmacists across five FMRPs in four states were invited to track their interactions with physician residents. EBM resources, skills, and targeted milestone data were collected. Pharmacists also quantified their nonpatient care contributions to EBM. Of the 16 clinical pharmacists invited, 16 (100%) participated in the October and 12 (75.0%) in the March collection period. A total of 598.9 half days over 2 months (42 working days) of available teaching time were reported. The tracking tool captured 1,253 EBM teaching encounters with a total average of 2.1 encounters per half day. Of those encounters, point-of-care references were most commonly used (63.7%) and "apply" was the most common EBM skill taught (83.8%). The most commonly tracked milestone was Medical Knowledge 2 (75.3%) at Level 2. Nine out of 10 faculty pharmacists included in this study reported performing the following roles: preceptor (100%), lecturer (89.9%), provider (77.8%), expert/consultant (77.8%), health care team (66.7%), and other (11.1%). Faculty pharmacists also reported directly evaluating milestones for physician residents through: committee work (44.4%), resident evaluations (77.8%), and rotation evaluations (77.8%). As FMRPs strive to meet ACGME EBM-related competencies, clinical pharmacists across multiple sites demonstrated contributions to teaching EBM in medical resident education. Using a nonphysician faculty for this purpose may provide an example for other FMRPs.

Point-of-Care Ultrasound in General Surgery Residency Training: A Proposal for Milestones in Graduate Medical Education Ultrasound.

The use of point-of-care ultrasound (US) in the clinical setting has undergone massive growth, although its incorporation into training and practice is variable. Surgeons are interested in using point-of-care US and can incorporate it effectively into clinical practice. However, the current state of point-of-care US training in general surgery is inadequate. The Accreditation Council for Graduate Medical Education introduced the Milestones Project to evaluate resident and fellow performance. Emergency medicine is the only specialty with a point-of-care US milestone. We have successfully implemented a US training program into our general surgery residency curriculum and now propose milestones in point-of-care US for all general surgery residents.

Results from the Field: Development and Evaluation of a Psychiatry Residency Training Rotation in Cognitive-Behavioral Therapies.

The goal of this project was to develop and evaluate a new residency training rotation focused on cognitive-behavioral therapies (CBT) and to assess outcomes across multiple domains. Data are presented from 30 psychiatry residents. Primary learning-related outcomes included content knowledge, self-efficacy, and attitudes and behavioral intentions towards evidence-based psychotherapies (e.g., CBT). Residents reported significant increases in CBT knowledge, CBT-specific self-efficacy, overall psychotherapy self-efficacy, belief in patient benefit from CBT, and behavioral intention to use CBT. However, there were almost no changes in attitudes towards evidence-based practice more broadly, with one significant finding showing an increase in skepticism towards such practices. This empirically based example of training program development, implementation, and evaluation appears largely successful and represents one approach for addressing the CBT competency goals outlined by the Accreditation Council for Graduate Medical Education (ACGME) and Milestone Project Guidelines. Despite these initial, positive findings, conclusions should be tempered by limitations of the project design (e.g., the lack of comparison group, absence of objective measures of resident behavioral change, or assessment of the effect of such changes on patient outcomes). Findings highlight the need for continued development and evaluation of training methods in CBT for residency programs.

Unaccompanied Latino Minors, Immigration, and Mental Health: An Opportunity for Advocacy

Child and adolescent psychiatrists (CAPs) have a duty to be patient advocates in multiple ways. Indeed, the Child and Adolescent Psychiatry Milestone Project’s inclusion of advocacy within the systems-based practice competency recognizes this role as a core element of practice. However, while training programs focus on teaching advocacy for individual patients’ needs, such as securing insurance coverage for medical treatment or obtaining school accommodations, preparedness in public policy and social justice domains is often limited. Accordingly, the current president of the American Academy of Child and Adolescent Psychiatry (AACAP), Dr. Gregory Fritz, has highlighted advocacy as one of four areas that his presidential initiative on integrated care will tackle.1 The main goals of this article are to discuss the role of CAPs in advocacy efforts as they apply to the vulnerable population of unaccompanied children from Central America, to discuss ways in which CAP training programs can promote the special interests and advocacy efforts of their trainees, and to encourage all CAPs to consider ways to amplify their voices and roles as children’s advocates.

Integration of Hands-On Team Training into Existing Curriculum Improves Both Technical and Nontechnical Skills in Laparoscopic Cholecystectomy

Nontechnical skills are an essential component of surgical education and a major competency assessed by the ACGME milestones project. However, the optimal way to integrate nontechnical skills training into existing curricula and then objectively evaluate the outcome is still unknown. The aim of this study was to determine the effect laparoscopic team-based task training would have on the nontechnical skills needed for laparoscopic surgery. 9 PGY-1 residents underwent an established training curriculum for teaching the knowledge and technical skills involved in laparoscopic cholecystectomy. Initial training involved a didactic session, expert-led practice on a porcine model in a simulated operating room and laparoscopic skills practice on a virtual reality trainer. Residents then performed a laparoscopic cholecystectomy on the same porcine model as a preintervention test. Three to four months following this, residents were subjected to specific nontechnical skills training involving 2 simple team-based laparoscopic tasks. They then practiced a further 4 to 6 hours on the virtual reality trainer. A repeat postintervention laparoscopic cholecystectomy was then performed 3 to 4 months after nontechnical skills training. Both the preintervention and postintervention laparoscopic cholecystectomies were audiovisually recorded and then evaluated by 2 independent surgeons in a blinded fashion. Technical skills were assessed using objective structured assessment of technical skills (OSATS) and a technique specific rating scale (TRS) that we developed for laparoscopic cholecystectomy. Nontechnical skills were assessed using nontechnical skills for surgeons (NOTSS). Residents also completed a survey at the beginning and end of the training. Tertiary care, university based teaching institution. A total of 9 general surgery residents at the intern level. The mean OSATS score improved from 13.7 ± 1.24 to 26.7 ± 0.31 (p < 0.001), the mean TRS score improved from 6 ± 0.46 to 13.1 ± 0.36 (p < 0.001) and the mean NOTSS score improved from 21.7 ± 1.83 to 36.3 ± 0.87 (p < 0.001) following the training. There was a strong correlation between OSATS and NOTSS scores (Pearson's R = 0.98) and TRS and NOTSS (R = 0.94). The inter-rater agreement was 0.79 for NOTSS, 0.9 for OSATS, and 0.82 for TRS. Following completion of the training, residents self-reported improvements in exchanging information (p < 0.01), coordinating activities (p < 0.01) and coping with pressure in the operating room (p < 0.001). Simple, team-based nontechnical skills training for laparoscopic cholecystectomy that was separate from technical skills training led to a sustained increase in residents' nontechnical skills 3 to 4 months after training. This was associated with a self-reported improvement in many nontechnical skills based on resident survey. Based on these results, we recommend that such designated nontechnical skills training is a valid alternative to other methods such as coaching and debriefing. We, therefore, plan to continue our efforts to develop team-based simulation tasks aimed at improving nontechnical skills for multiple surgical modalities.

The Impact of Using Mean Versus Mode When Assessing Resident Competency.

The Accreditation Council for Graduate Medical Education Milestone Project was implemented in 2014 to standardize assessments and progression of residents. While it is recommended that milestones not be used as tools for direct assessments of resident competency, many programs have used or adapted milestone tools for this purpose. We sought to explore use of the most frequent milestone level at which a resident was evaluated (ie, the mode), and compared this to the standard practice of using the arithmetic mean for summarizing performance. We reviewed all Family Medicine Milestone evaluations from 1 program for the first 2 academic years of milestone implementation. Mean and mode scores were calculated across 24 unique residents, 841 evaluation forms, and 5897 measurements. The proportion of overestimation errors (where the mean is at least 0.5 larger than the mode) and underestimation errors (where the mean is at least 0.5 less than the mode) were then compared across resident training year and subcompetency. For the 24 residents, an estimation error occurred in 175 of 792 of the comparisons (22%). Of these errors, 118 (67%) were overestimation errors. First-year residents accounted for 55% (96 of 175) of all estimation errors. All subcompetencies had some estimation errors, with 6 having greater than 5%. If the trend for using the milestones as stand-alone assessment tools is to continue, aggregating data by using frequency distributions and mode would be a more stable and appropriate approach given their nominal or, at best, ordinal nature.

Trainee Involvement in Patient Care: A Necessity and Reality in Teaching Hospitals.

Trainee Involvement in Patient Care: A Necessity and Reality in Teaching Hospitals.

PD58-10 QUALITY IMPROVEMENT EDUCATION AND PARTICIPATION IN UROLOGY RESIDENCY PROGRAMS: PRELIMINARY SURVEY RESULTS OF PROGRAM DIRECTORS

You have accessJournal of UrologyGeneral & Epidemiological Trends & Socioeconomics: Quality Improvement & Patient Safety I1 Apr 2017PD58-10 QUALITY IMPROVEMENT EDUCATION AND PARTICIPATION IN UROLOGY RESIDENCY PROGRAMS: PRELIMINARY SURVEY RESULTS OF PROGRAM DIRECTORS Justin Ziemba, Brian Matlaga, and Christopher Tessier Justin ZiembaJustin Ziemba More articles by this author , Brian MatlagaBrian Matlaga More articles by this author , and Christopher TessierChristopher Tessier More articles by this author View All Author Informationhttps://doi.org/10.1016/j.juro.2017.02.2625AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookTwitterLinked InEmail INTRODUCTION AND OBJECTIVES One of the key physician competencies outlined in the Urology Milestone project is engagement in quality improvement (QI). Despite this mandate, little is known about how QI education and participation by residents is integrated into their training. Therefore, we performed a national survey of urology residency program directors (PD) in partnership with the Society of Academic Urologists (SAU). METHODS A 37-item electronic survey was developed to assess QI education, mentorship, and participation within a urology residency program. The survey was tested by an assistant program director and revised based on the feedback received. The survey was sent via email (11/1/2016) to all PD affiliated with the SAU (n=116; 94% of ACGME programs). Included in this analysis are only the completed responses received from the initial survey invitation at 1 week (censor date 11/7/2016). RESULTS A total of 22 respondents returned a completed survey for a response rate of 19% (22/116). Table 1 outlines program characteristics. All PD reported they participate in direct patient care, but 36% (8/22) also listed QI leader and educator as an additional role. Only 27% (6/22) of PD have received formal training in QI methodology, but 59% (13/22), 55% (12/22), and 32% (7/22) are directly involved in QI education, serve on a QI committee, and lead a QI team, respectively. Only 45% (10/22) of programs have a formal curriculum for teaching QI to residents with 90% (9/10) using didactic sessions focusing on Six Sigma (5/10; 50%) and root cause analysis (5/10; 50%) methodology. Table 2 lists PD attitudes towards QI education and participation for residents. The majority of PD (19/22; 86%) would like to see the American Urological Association develop a urology-oriented QI curriculum. CONCLUSIONS A minority of programs have QI education available for residents. However, PD agree that QI is an integral part of residency training, which should be promoted by our profession. Responses from additional PD will allow us to validate these initial trends. © 2017FiguresReferencesRelatedDetails Volume 197Issue 4SApril 2017Page: e1131 Advertisement Copyright & Permissions© 2017MetricsAuthor Information Justin Ziemba More articles by this author Brian Matlaga More articles by this author Christopher Tessier More articles by this author Expand All Advertisement Advertisement PDF downloadLoading ...

Program Director Perceptions of the General Surgery Milestones Project

As a result of the Milestones Project, all Accreditation Council for Graduate Medical Education accredited training programs now use an evaluation framework based on outcomes in 6 core competencies. Despite their widespread use, the Milestones have not been broadly evaluated. This study sought to examine program director (PD) perceptions of the Milestones Project. A national survey of general surgery PDs distributed between January and March of 2016. A total of 132 surgical PDs responded to the survey (60% response rate). Positive perceptions included value for education (55%) and evaluation of resident performance (58%), as well as ability of Milestones to provide unbiased feedback (55%) and to identify areas of resident deficiency (58%). Meanwhile, time input and the ability of Milestones to discriminate underperforming programs were less likely to be rated positively (25% and 21%, respectively). Half of PDs felt that the Milestones were an improvement over their previous evaluation system (55%). Using the Milestones as competency-based, developmental outcomes measures, surgical PDs reported perceived benefits for education and objectivity in the evaluation of resident performance. The overall response to the Milestones was generally favorable, and most PDs would not return to their previous evaluation systems. To improve future iterations of the Milestones, many PDs expressed a desire for customization of the Milestones' content and structure to allow for programmatic differences.

Competency milestones for medical students: Design, implementation, and analysis at one medical school

Competency-based assessment seeks to align measures of performance directly with desired learning outcomes based upon the needs of patients and the healthcare system. Recognizing that assessment methods profoundly influence student motivation and effort, it is critical to measure all desired aspects of performance throughout an individual’s medical training. The Accreditation Council for Graduate Medical Education (ACGME) defined domains of competency for residency; the subsequent Milestones Project seeks to describe each learner’s progress toward competence within each domain. Because the various clinical disciplines defined unique competencies and milestones within each domain, it is difficult for undergraduate medical education to adopt existing GME milestones language. This paper outlines the process undertaken by one medical school to design, implement and improve competency milestones for medical students. A team of assessment experts developed milestones for a set of focus competencies; these have now been monitored in medical students over two years. A unique digital dashboard enables individual, aggregate and longitudinal views of student progress by domain. Validation and continuous quality improvement cycles are based upon expert review, user feedback, and analysis of variation between students and between assessors. Experience to date indicates that milestone-based assessment has significant potential to guide the development of medical students.