Taking Stock and Looking Ahead: Evolution of Accreditation Feedback for Simulation Centers Over 8 Years Using Epistemic Network Analysis.

Abstract

Since 2005, the American College of Surgeons Accredited Educational Institutes has provided accreditation of surgically focused simulation centers with the added benefit of identifying best practices defined as areas far exceeding the accreditation standards or novel methods of advancing high-quality, impactful education. This study aimed to examine the evolution of the best practices observed by accreditors during site visits over the 8-year period. Accreditation included the completion of an application form followed by a site visit by a simulation expert and review of all materials by an accreditation committee to identify areas out of compliance along with areas far exceeding accreditation standards. These are termed "best practices." To evaluate the evolution of accreditation feedback and embedded associations, the compiled list of 337 best practices identified from all 247 site visits over an 8-year period was analyzed and visualized using epistemic network analysis, a quantitative ethnographic technique for modeling the structure of connections in qualitative data. The overall association network of the data indicates that the strongest associations were between assessment, curriculum development, faculty development, research, and teaching methods, demonstrating a highly interconnected model of accreditation feedback. Best practices evolved from an early focus on teaching methods, faculty, and curriculum development to more advanced educational topics including assessment, research, resources, and overall center governance. Distribution of associations also increased over the 8-year period with more nuanced and interconnected statements demonstrating higher-level feedback including explanations, contributing factors, impact on other areas, and, in some cases, recommendations to share best practices outside the organization. The epistemic network analysis of this 8-year database of simulation center feedback provides a novel perspective on an organization and the evolving field of simulation from an optional to essential modality in healthcare professions education.