Use of qualitative analysis to supplement program evaluation of a faculty development program

Abstract

Introduction: Generally for Program Evaluation of workshops, a structured questionnaire is used to measure learner satisfaction and learning and it yields results in quantitative terms and if there are open-ended questions, it yields a long list of responses which the evaluator finds it difficult to use. Hence, we used qualitative research methods to supplement it for a holistic understanding of the quality and extent of their learning. Materials and Methods: A faculty development program was used opportunistically to gather data using open-ended questions designed to elicit thick descriptions about their learning. Using the Grounded Theory approach where the data drives further analysis, data analysis was done by faculty in three groups using inductive reasoning leading to emergence of themes and categories of responses. The commonalities in the themes that emerged independently in the three groups as well as member checking to seek their agreement with the group's observation of the primary data were used to ensure validity of the findings. Results: Themes and categories of new learning in terms of knowledge, skills, attitudes and processes emerged which gives a better understanding of the extent of the learning and its applicability to future professional tasks. The open-ended questions encouraging participants to give thick description of their learning yielded many quotable quotes which indicate the quality of their learning. Conclusion: The use of open-ended questions and qualitative research method of data analysis using the Grounded Theory approach helps in holistic understanding of the focus and quality of learning that is not usually possible through quantitative research methods. Hence, it is recommended that when understanding of the extent and quality of learning is important, traditional method of program evaluation should be supplemented with qualitative research methods as they are more effective in convincing decision makers.