Early Impressions of the CLER Program: A Survey of the Designated Institutional Official Community.

Abstract

The Accreditation Council for Graduate Medical Education (ACGME) established the Clinical Learning Environment Review (CLER) Program in 2012. The purpose of the CLER Program is to provide the clinical learning environments (CLEs) that serve as the sites for the education of more than 120 000 residents and fellows in ACGME-accredited programs with periodic feedback that addresses 6 focus areas: patient safety, health care quality, care transitions, supervision, duty hours/fatigue management and mitigation, and professionalism.1,2 The aim is to present information to promote discussions and actions to optimize the educational experience of residents and fellows in the CLER focus areas and to improve patient care.3 CLER staff surveyed designated institutional officials (DIOs) from the initial round of CLER visits to assess early perceptions of the CLER Program and its initial effects.CLER site visits are structured to involve interviews with graduate medical education (GME) and CLE executive leadership, the organization's leaders in patient safety and health care quality, residents and fellows, faculty members, and program directors, along with observations and interview data gathered during walking rounds of the site's clinical units.1,2 At the conclusion of the visit, GME and CLE executive leadership receive an oral report of findings in the 6 focus areas, followed by a written report with a detailed set of observations to help target improvement efforts.As part of a systematic effort to improve the CLER Program, an online survey was distributed in September 2015 to DIOs of the 297 ACGME-accredited sponsoring institutions (SIs) with 3 or more ACGME-accredited core programs that had an initial CLER visit (September 2012–March 2015). The purpose was to assess perceptions of the first round of CLER visits and the changes made by each institution to improve its CLE.Requests for participation in a 14-item online survey were sent via e-mail, followed by 3 reminders. The survey included closed and open-ended questions on the value of the written CLER report; institutional engagement in the CLER focus areas; changes made to improve the CLE since the CLER visit and resources needed for this; valuable aspects of the CLER Program; and suggestions for improving the visit process.Analysis of quantitative data included frequency counts, percentage distributions, and 1-way analysis of variance was used to compare and identify differences in means by year of visit, regional location, number of programs at institution, and responding DIOs' years in their role. Statistical analyses were conducted using SPSS Statistics version 22.0 (IBM Corp, Armonk, NY). Analysis of qualitative data involved coding to identify themes using NVivo 10 (QSR International, Melbourne, Australia). The frequency with which each theme occurred is reported as percentages.A total of 231 DIOs responded to the survey (78% overall response rate). Data for 7 respondents were excluded from the analysis due to incomplete responses, and an additional 29 responses were excluded because the respondent was not the DIO at the time of the CLER visit. This yielded 195 usable responses (66% final response rate). Table 1 presents general characteristics of responding DIOs and their SIs. Fifty-six percent of the institutions visited had 18 or more ACGME-accredited programs, and 51% of responding DIOs had been in their position for 6 or more years.Forty-five percent of DIOs (88 of 195) reported their institution's GME community was moderately engaged or very engaged in developing strategies in the CLER focus areas before the first CLER visit; 84% (163 of 195) agreed or strongly agreed that the GME community was more involved in improving 1 or more CLER focus areas as a result of the visit (see table 2 for mean ratings).GME engagement before the initial visit was significantly associated with the year of the first visit. Compared to DIOs who had an initial visit in 2014, those with a visit in 2012 and 2013 were less likely to perceive their GME community was engaged. Among the SI- and DIO-related characteristics assessed, none were significantly associated with GME engagement after the first visit. Eighty-five percent (22 of 26) of DIOs who reported that their GME community was slightly engaged or not at all engaged in developing strategies in the CLER focus areas before the first CLER visit agreed or strongly agreed that their GME community was more involved as a result of the first visit.Overall, 85% (165 of 195) agreed or strongly agreed that the written report provided information that helped their institution target areas for improvement in their CLE.When asked about changes made in the CLE since the initial CLER visit, 8 general themes emerged from 382 lines of coded text, shown in figure 1. Themes included (1) increased focus on patient safety and quality improvement (QI); (2) increased focus on resident supervision and improving mechanisms for assessing resident and fellow competency to perform clinical procedures and dissemination of this information; (3) structural and other changes by clinical sites to address the 6 focus areas, such as the establishment of committees to integrate the focus areas into GME and CLE initiatives; and (4) improving care transitions, most often through standardized approaches. Other themes included changes in GME and CLE executive leadership, duty hours and fatigue management and mitigation strategies, efforts aimed at enhancing professionalism, and efforts to improve data collection and reporting. Within the most prominent theme—enhancing patient care quality and safety—specific initiatives included an increased focus on QI and patient safety at resident orientation and through formal curricula; increased resident and fellow participation in QI and patient safety initiatives; and increased education of residents and fellows in the use of patient safety reporting systems.In relation to resources needed to improve the learning environment, the 3 main themes are shown in the box. Within the theme of professional development, DIOs mentioned the importance of developing faculty competency and expertise in the CLER focus areas, such as techniques to improve patient safety so that effective role modeling and reinforcement can occur. They also commented on the need for protected time to attend faculty development sessions, pursue research, champion innovative teaching, and ways to effectively educate, mentor, and assess resident learning as part of successful educational efforts.The second theme pertained to the need for faculty and support staff in the institutional GME office to engage in efforts to address the CLER focus areas. Ready access to data or information technology support across the institution was the third theme, including systems that linked patient outcome data to institutional patient safety and QI efforts.In response to the question about aspects of the CLER Program that DIOs viewed as valuable, 7 general themes emerged and are shown in figure 2. The benefit of receiving formative feedback to identify areas for improvement related to the CLER focus areas was the leading theme. DIOs also reported that the visit helped increase awareness of the role of the GME enterprise in enhancing quality and safety in the CLE, focused attention on the importance of the CLER focus areas, and increased collaboration and engagement between GME and CLE executive leadership in improving resident and fellow training in the focus areas.When asked about suggestions for improving the CLER site visit process, DIOs expressed the need for more time to prepare for the visits. In addition, they suggested adding comparative data in the written report and more flexibility on who can attend the group meetings.The results of the survey suggest a positive perception of the CLER Program. As a result of the initial CLER visit, DIOs reported that the GME community is more involved in the focus areas, particularly in CLEs where there was little or no engagement prior to the visit.In terms of outcomes, the findings indicate that most improvement efforts are currently concentrated in the areas of patient safety and health care quality, with emphasis on educational activities to increase awareness of initiatives in these areas, as well as improving supervision and care transitions. These changes reflect how CLEs have utilized the information from the site visits to identify and prioritize areas for improvement in the 6 focus areas.Consistent with the aims of CLER, DIOs appeared to recognize the value of formative feedback to improve the CLE. They also acknowledged that the program set expectations for the integration of GME within the CLE to improve both resident and fellow training and patient care. DIOs indicated that an important value of the CLER Program is that it underscores the engagement of both GME and CLE executive leadership—elevating the dialogue on how best to improve resident and fellow engagement in the CLER focus areas.The results also provide insights into areas for improving the CLER Program. Given the current focus on QI and patient safety, the CLER Program will need to identify ways to help drive improvement in the other focus areas. The survey also identified opportunities to improve the site visit experience to enhance engagement between GME and CLE executive leadership, increase interprofessional dialogue, and address operational issues such as scheduling.As with all studies, there are limitations that should be considered. The findings are based on visits to larger SIs with 3 or more core specialty programs and may not generalize to smaller SIs. In addition, institutions with visits during 2012 and 2013 may have had more time to experience change. Finally, although the survey was administered anonymously, a study conducted by investigators external to ACGME may have produced different findings.The DIO community is reporting changes that are related to the formative feedback from the CLER Program and the resulting learning. Over time, the CLER Program will continue to assess how these changes translate into actions that improve the quality of the CLE and the impact of this on resident and fellow education and, ultimately, patient care. The findings highlight challenges that exist to improve the CLE and the ways in which to improve the quality of the CLER visit experience, as well as offer opportunities to maximize learning from the initial implementation of the CLER Program in ongoing program refinement.