A Standards-Based Checklist for Program Evaluation as Scholarly Activity.

Abstract

Educators in graduate medical education (GME) regularly make decisions about adding, deleting, or revising some aspect of a curriculum, rotation, or other program activity. At the core of each decision is the activity's value to the learners, faculty, program, and sponsoring institution, and/or to meeting accreditation/licensure requirements. When these decisions are informed by systematic program evaluation, the results may be generalizable and have value to others in GME beyond their local worth. Yet how does an educator know if their work might be relevant to others or fulfill scholarship criteria? These are particularly vexing challenges for evaluation, as every program is local and has unique settings, trainees, faculty, and resources. Thus, disseminating evaluation as scholarship is a balancing act: staying true to the local nature of the program while addressing concerns of skeptics who may say, “That's not relevant…it wouldn't work here.”Applying standards of program evaluation can tip the balance toward becoming a scholarly activity. Other Rip Outs1,2 in this series outlined the American Evaluation Association (AEA) standards (accuracy, feasibility, integrity [ie, propriety], and utility)3 and suggested their application to program planning and implementation. Viewing these same standards in a checklist format can guide evaluation as a scholarly activity for authors and reviewers. Reminder: standards apply to evaluation, not to the project (eg, asking if the evaluation is feasible, not if the project is feasible). Standards and sample items include:As evaluators, readers, and editors, we like checklists ordered to match the standard introduction/aim, methods, results, discussion/conclusions (IMRD) framework. However, evaluation standards underpin the entire program evaluation endeavor; thus, one standard can—and often should—appear in multiple sections. See online supplementary data for the complete Evaluation as Scholarly Activity Checklist by AEA Standards.