Evaluation of Participants' Reaction and Learning in a Taught Analytics and Modelling Academy Program in U.K.‘s National Health Service

Abstract

Recent research has highlighted the need to invest in the development of healthcare analytics capability. However, the contents of such programs and how they should be delivered to maximize the learning outcome are unclear. In this paper, we provide insights into the learning within the first two cohorts of modelling fellows successfully trained in an analytics and modelling academy run within the National Health Service (NHS) Wales, U.K. The participants followed a taught healthcare analytics and mathematical modelling program tailored for senior staff members including managers and clinicians. We build our learning evaluation framework on Kirkpatrick's training evaluation model and participants filled in questionnaires with respect to their level 1 (reaction) and level 2 (learning) experience after each module. In addition, we asked the participants about their self-assessments during three time points in the program. The qualitative feedback results revealed that the participants appreciate the learning and reflect where they could use the new developed skills in practice. They also provided useful suggestions for improving the program. The participants' aggregated quantitative self-assessments show a statistically significant increase in competence. In conclusion, this may lead to a behavior change in applying the methods on the job (level 3) and, ultimately, improve level 4 outcomes through analytics-driven healthcare improvement.