Faculty Development for Milestones and Clinical Competency Committees.

Abstract

The goal of postgraduate training is the development of competency necessary for independent practice. To achieve this goal, the Accreditation Council for Graduate Medical Education (ACGME) in 2013 implemented the Milestones assessment framework and mandated the development of clinical competency committees (CCCs), tasked with biannual review of evaluation and assessment data of trainees.1 The CCC, which consists of a chair and committee members, plays a critical role in assisting the program director in decision-making regarding resident or fellow advancement through the training period. Importantly, the committee makeup may span junior and senior faculty and include interprofessional representation, thus providing a spectrum of input.1To date, most of the literature regarding CCCs focuses on optimizing CCC function in terms of workflow and decision-making. Best practices for CCC function have been proposed, including use of varied assessment tools, structured meetings, and diverse members.2,3 Other work has outlined strategies for effective group decision-making necessary for CCC function.4–6 Ongoing faculty development has been proposed as an essential component of CCC function, yet little is known about which particular elements are necessary to optimize organization of meetings, decision-making, and to effectively utilize assessment processes.In order for the CCC to optimally operationalize the Milestones for individual learner development, members of the committee require intentional faculty development. This must span multiple domains important for the CCC process, including knowledge of assessment (and the use of the Milestones), understanding of group decision-making, awareness of how biases interplay with these factors, and the effect of their decisions on patient care (Box). While it remains unknown how often and who should develop such faculty development sessions, we would advocate for at least an annual review of these topics given faculty turnover and need to maintain a shared mental model of best practices. Faculty development remains essential for both new and longstanding members, as participants will likely have different levels of understanding of the assessment process and Milestones themselves.2We therefore aim to provide guidance for specific areas of faculty development for CCC members, including strategies for novel approaches to professional development for the committee. This includes: (1) the purpose and structure of the CCC itself; (2) the strategies, strengths, and weaknesses of the assessments employed by the program; and (3) the development of a shared mental model of the Milestones of the relevant specialty. Beyond this, the CCC chair will also need to understand (4) the best practices of an effective group discussion.7Faculty need to understand the purpose of the CCC meetings, which often employ 1 of 2 strategies: problem identification, where the committee focuses on identifying learners who may require additional remediation, or a developmental approach, where the committee discusses all learners to identify next steps for advancement.3 Members also need to understand the process of decision-making, including who will make the recommendations regarding remediation.In addition to understanding the intent of the Milestones, committee members must also be aware of the program's evaluation process (as related to each Milestone). Knowledge of the individual assessment methods and awareness of the data synthesis for each Milestone will improve a shared mental model and ideally minimize differences in individual interpretation of assessment data.6 This will also allow the committee to identify times when the assessment data is difficult to interpret requiring further group discussion.8Committee members should not only understand each of the relevant Milestones but also have a shared model of what a competent trainee looks like at their institution.9 It is evident that competencies are developed differently in different contexts and at different institutions.10 Therefore, a shared mental model is important for each CCC to minimize variation in how each member interprets the same assessment data.Effective committee groups understand the purpose, structure, and process of their work.4 However, sometimes groups encounter incomplete or otherwise difficult to interpret assessment data; therefore, group discussion is needed to draw a conclusion. The chair of the committee needs to lead the discussion and approach how decisions are made transparently. The chair should also be cognizant of the member composition of the group.4 A homogenous group is unlikely to present different views or opinions, which may be necessary in drawing fair conclusions.Additional faculty development topics for the CCC chair may be helpful in maximizing success of the committee. It is incumbent upon the chair to refine their own ability to synthesize feedback as well as interpret and gauge performance in specialty-specific Milestones. General leadership skills, such as how to effectively lead a committee or run a meeting, remain paramount.11,12 It is important for the CCC chair to adapt their mindset from the frame of educator to that of the “educator-leader.” The CCC chair plays a pivotal role in ensuring that the committee functions through fair and informed decision-making processes based on the data and facts, rather than emotional decision-making.Beyond this unique CCC training for the chair and members, all faculty educators must understand the intersection of biases on the assessment process. It is well-known that biases can manifest in a variety of ways through our traditional mechanisms of assessment. For example, in graduate medical education (GME), this has been highlighted in several settings showing differential Milestone achievement based on trainee gender,13–15 as well as gender differences in narrative evaluation.16 Similarly, standardized patient assessments, which are often included in CCC discussions, have highlighted differential findings by gender, race, and ethnicity.17 Knowledge of such biases and the potential ramifications on Milestones ratings, narrative evaluation, remediation referrals, and progression decisions is imperative for faculty development in the CCC process.Faculty development using bias reduction workshops or training, such as those employing the Implicit Association Test, could represent an avenue to initiate discussions,18 as these training programs have shown to be effective in mitigating biased decision-making in other settings.18–20 The potential quantitative and qualitative differences in assessment should be considered in recurring CCC reviews. Additionally, faculty participants should be versed in the potential impact of system-wide and personal biases on trainee assessments. Beyond an individual training level, CCC structure, including the diversity of CCC members, as well as process measures, including program assessment and trends, should be regularly reviewed for biased tendencies.2There are a multitude of topics important in faculty development for CCC chairs and members and implementation often requires novel solutions. One such approach was recently implemented within the MedStar Health GME Consortium in the Baltimore, MD and Washington, DC region, which includes more than 70 ACGME-accredited programs. The consortium conducted a half-day faculty development retreat specifically for CCC chairs. Topics addressed included a primer on evaluation of competence, a panel discussion of CCC best practices (for varied sizes of programs), small group discussions on CCC decision-making processes, and strategies for running effective meetings. The program also allowed for identification of distinct needs for various CCC stakeholders, as outlined in the Table. These sessions allowed for shared knowledge, highlighted knowledge gaps, and identification of distinct faculty training needs for success of CCCs. Such an approach allowed for sharing of best practices, strategies to overcome real or perceived challenges (such as time constraints to effective discussion in larger scale programs), and dissemination of knowledge throughout the sponsoring institution.Overall, the optimal functioning of a CCC requires dedicated faculty time, education, and investment in faculty development. CCCs form a critical component of GME assessment and play a pivotal role in ensuring readiness for independent practice. Given this critical role, we advocate for an appropriate fraction of protected time (or productivity) from clinical or other administrative duties for CCC chairs and similar key leaders to accomplish the important responsibilities of the role.We have highlighted some overarching themes of essential faculty development for CCC success, but recognize that the needs are varied based on role within the CCC, experience level, and size of the program, among other considerations. It is critical to acknowledge differing faculty development needs across levels of experience of faculty (junior versus senior members), as well as the unique training needs of interprofessional members. Additionally, future scholarship is needed to determine the optimal timing, content, and subsequently the impact of such faculty development on CCC processes. We believe that the continued development of expertise for both the CCC chair and its members remain critical to the success of the Milestones assessment system.