Measuring the Accuracy and the Potential for Standardizing Students’ Evaluations of Teaching: The Use of Measurable Bench-Marking Questions

Abstract

Student evaluations of teaching (SETs) are widely used to quantitatively assess the competence of university lecturers. SETs can be used formatively to direct lecturers in improving their teaching, and more summatively in determining the suitability of a candidate for employment, confirmation in post or tenure, promotion, and performance-based salary adjustment or reward. The validity and suitability of SETs remains heavily contested, yet few papers provide workable recommendations to improve the validity of the tool or the interpretation. This results in perpetuating distrust for SETs, and, in cases, lecturers behaving to artificially enhance their scores. We present a method to assess the accuracy of SETs, either informally by the lecturer in understanding the outputs, or formally by the administrative body that distributes SETs and their scores. We provide recommendations for standardizations of score outputs using bench-marking questions that can highlight biased responses, and to adjust output scores and assessment selections accordingly.