Exploring the trustworthiness and reliability of focus groups for obtaining useful feedback for evaluation of academic programs.

Abstract

At Shifa College of Medicine, evaluation is an integral part of the curriculum. We used focus groups as a tool to obtain in-depth information regarding students' experience with the new integrated, system- based curriculum. The purpose of this study was to explore the usefulness of focus groups in identifying important issues for curriculum improvement and to explore the trustworthiness and representativeness of data obtained through this strategy. In 2012, we used focus groups to seek feedback from students regarding their experience with the integrated curriculum. One course of each of the three preclinical years was selected for this purpose. Three parallel focus groups were conducted for each selected course. Each focus group was audio recorded, and the moderator and a volunteer student took additional notes during the session. The audio recordings were transcribed and data obtained from the three sources were analyzed, coded, and categorized independently by three investigators. Both manifest and latent themes were identified, using an inductive approach. Final agreement on themes was reached by comparison of the independently done thematic analysis by the three researchers. Reliability of data was established by comparing responses from the three parallel focus groups of each course. Trustworthiness of inferences was ensured by multiple coding, audit trail and member checking with focus group participants who reviewed the themes for validity. Most of the data on students' perceptions of their courses from each of the parallel groups were in agreement: Similar themes were seen within groups of the same class as well as across the three preclinical years. Focus groups can be a useful tool for collecting trustworthy and reliable information through a process that promotes interaction among student participants. They can support quantitative data from students and be used to support curriculum reform.