Toward understanding and quantifying halo in students’ evaluation of teaching

Abstract

Despite its name, the halo effect in student evaluation of teaching (SET) response is not mystical. Halo in SET results from psychological processes that undermine SET validity, particularly for summative evaluation (pay decisions, etc.). These processes span psychological concepts of cognition, motivation and affect. This paper demonstrates that halo signals serious trouble for SET data, contrary to the conclusion in Cannon and Cipriani (2022), to which this article replies. Re-analysis of those authors’ findings, conducted for the present paper, reveals both more halo and less validity than Cannon and Cipriani reported. For re-analysis of their factor analysis, a practical, generalizable approach to gauging halo in various settings is proposed. This approach revealed consistently high apparent SET halo across various decades and geographical locations of SET use. For re-analysis of their multiple regression, an approach to deciphering individual variables’ effect sizes is provided for designs like Cannon and Cipriani’s. Implications discussed for summative use of SETs include issues of veracity of SET confidence intervals. In contrast to summative use, formative use (toward performance improvement) suffers less from halo because non-instructional factor influences are somewhat constant for individuals.