Qualitative Analysis of the Impact of Changes to the Student Evaluation of Teaching Process

Abstract

Objective. To determine how changes to the student evaluation of teaching (SET) survey instrument and process at a college of pharmacy contributed to improved student response rates and to understand how the process could be further refined. Methods. Pharmacy students from the class of 2018 who had participated in both the old and new SET process were recruited to participate in one of four focus group interviews. An inductive approach was used for data collection and analysis. A focus group guide was created based on two major domains: comparing changes between the old and new SET process and survey form, and determining how the new SET process could be further refined. Results. In South Jordan, UT, six students participated in one of the focus groups and seven students participated in the other focus group. In Henderson, NV, seven students participated in each of the two focus groups. Twenty-seven total students participated in the four focus groups across two campuses. Students stated that reducing the number of questions on each SET survey instrument and using a 5-point rather than a 10-point Likert scale were positive changes. The changes also motivated them to complete the surveys, which improved overall response rates. Although students reported that the monetary incentive (contributions toward the cost of the class banquet) that had been added to the new SET process was a strong motivator, the incentive itself would have likely been insufficient without the other changes. Several participants stated that receiving feedback from faculty members on changes made to teaching materials based upon previous student evaluations was also an important motivator for students to continue completing the surveys. Conclusion. Students identified several motivators for SET participation. Improving the process for survey completion is essential to improve response rates to more accurately represent the feedback of the entire student body. Additionally, the evaluation process must ensure that the data gathered are robust, accurate, and insightful, to be of good use of student and faculty time.