Bias hypotheses under scrutiny: investigating the validity of student assessment of university teaching by means of external observer ratings

Abstract

To advance the discussion on the validity of student evaluations of university teaching, student ratings of two teaching dimensions – student involvement and rapport – were compared with corresponding observer ratings. Seven potential bias variables were tested with regard to their impact on the students’ teaching assessment: three teacher characteristics (first impression, enthusiasm, humour) and four student characteristics (prior interest, expected grades, study experience, class attendance). Bias was defined as an impediment of the students’ assessment of teaching on course level. By means of bivariate correlations with course averages and two-level latent moderated structural equations, data of 1,716 students in 80 courses were analysed. Results showed that all three teacher characteristics were genuinely connected to rapport, and even explained variance of the student-rated variable when controlling for observer-rated rapport. The assessment of student involvement was not modified by the teacher characteristics except for teacher enthusiasm, which affected the student evaluation when controlling for observed involvement and, moreover, moderated the relation between the observed and the student-rated variable. For the examined student characteristics, no biasing effects were found – neither on rapport nor on student involvement.