Unreliability in Student Evaluation of Teaching Questionnaires: Focus Groups as an Alternative Approach

Abstract

The conventional method used to measure teaching effectiveness is the student evaluation of teaching questionnaire (SET). Research on the validity and reliability of SETs is vast, though riddled with inconsistencies. The many “myths” of SETs are investigated and the incongruities are demonstrated. We hypothesize that the discrepancies in empirical studies come from misunderstanding and inappropriate actions by students. To address the complexity inherent in these problems, we suggest the use of focus groups as an alternative approach or complement to the standard SETs. A recommended format and guidelines for running classroom focus groups are provided. Institutional constraints and implementation concerns are addressed as well. This article lays the foundation for implementing a change in student assessment of teaching by proposing a method to compensate for bias in SETs, using focus groups as an evaluation tool, either as a stand-alone process or as a supplement to current methods.