Formative Evaluation of Using Action Learning in a Master of Medical Education Assessment and Measurement Course

Abstract

Background: The future success of any graduate or professional degree program is dependent upon continuous feedback provided by instructors and students. Various teaching models used by medical educators include didactics, problem/case-based learning, small/large group work, distance/online education, simulation, labs, and service/experiential learning. Action Learning is a process “that involves a small group working on real problems, taking action, and learning as individuals, as a team, and as an organization.” Medical school curricula usually begin with a mostly knowledge-based approach to learning the relevant science courses. While it may include some experiential learning, there is limited organized reflection. The idea inherent in Action Learning is “learn while doing” and “reflect on the experience.” This paper reports the process and outcomes of using the Action Learning Model (ALM) in teaching a master’s level assessment and measurement medical education class.Objective: The objective of this quality improvement education study was to ascertain students’ knowledge, skills, and attitudes demonstrated in conducting substantive evaluations using the ALM in a graduate medical education assessment and measurement course.Method: This study was a formative evaluation of a 16-week master’s level medical education assessment and measurement course. The curriculum included teaching the traditional knowledge, skills, and attitudes (KSAs) to conduct formative and summative evaluations in medical education. In addition, students learned applicable quality improvement skills. Specifically, they learned how to identify and work with valid customer (student) requirements, how to map and improve processes, and how to collect and analyze process data. Students were taught the KSAs while conducting a formative evaluation of the class as their major project. They evaluated the class they were taking while reflecting on the experience. In addition to the ALM, the course incorporated both the Bloom Taxonomy (a hierarchical framework for cognition and learning objectives) and the Kirkpatrick Model (a globally recognized method of evaluating the results of training and learning programs). The one-sample significance test was used to evaluate the median of the difference between the pre-and post-test groups. Descriptive statistics were also performed.Results: Nine students who were medical students, dental students, physicians, and simulation lab technicians participated in the course. Students learned medical education assessment and measurement of knowledge, skills, and attitudes (KSAs) and experienced the process of performing a formative evaluation. The post-test results for all students combined revealed that 277 of the 450 (61.6%) data points were greater than zero. A total of 139 data points showed no improvement and 34 results were worse than the pretest. Discussion: The ALM for teaching assessment and measurement in medical education can be challenging, but it may provide a more realistic and rewarding educational experience. The students gained a greater appreciation of the positive and negative aspects of using an experiential approach. Finally, the weekly formative surveys provided regular feedback that led to instructional improvements. With regards to medical education, action learning is best suited for students during the clinical phase of their education.