Measuring Faculty Skill and Effectiveness in Conducting Interactive Learning Sessions—a Proposed Instrument

Abstract

Medical educators have created approaches such as the “flipped classroom” to cultivate robust knowledge transfer with high-order thinking skills and retention. Hybrid approaches, with independent self-directed learning and large group interactive “application” activities, may also better foster learning. No standard instrument exists to rate the effectiveness of interactive teaching in large groups. This study developed an objective criterion-based instrument to measure the effectiveness of large group teaching by educators who engage with an audience that is pre-prepared by self-directed learning. We call such an educator a “Conductor of Interactive Learning” (COIL) and the interactive skillset, COIL skills. Based on needs assessment surveys of local and national end users, we designed a measurable standard-based instrument to evaluate effectiveness of a COIL. The instrument was designed to delineate measurable objective standards that characterize COIL skills and serve as a peer/self-assessment tool to identify individual COIL skills. After beta testing across institutional medical educators (n = 28), improved versions were presented to national experts (n = 12) and potential medical educator end users (three conference workshops) for constructive feedback on the quality and usefulness of the standards and the rating instrument. We developed standards that captured both interactive skills (n = 7) and non-interactive educational fundamentals (n = 9) for large group teaching. Using an iterative consensus building national process, we finalized a hybrid rating instrument consisting of these standards with anchors adjacent to merged 4-point Likert and visual analog scales. The standards and rating instrument for COIL skills could serve as an individualized guide for educator development.