Significant interpretation of small mean differences in student evaluations of teaching despite explicit warning to avoid overinterpretation.

Abstract

Student evaluations are a critical measure of teaching quality used for evaluation and scholarship of teaching. However, teachers and administrators tend to overinterpret small differences in student evaluation means. The current research consisted of 2 studies demonstrating that explicit warnings do not reduce overinterpretation. Study 1 showed that faculty (n 88) interpreted small, statistically nonsignificant differences in evaluation means between specific courses regardless of whether or not they received a warning. Study 2 showed that administrators (n 113) who receive a warning still judged instructional methods more negatively if the methods were associated with small reductions in evaluation means. These findings illustrate the importance of clearly identifying meaningful versus meaningless differences when using student evaluations.