Adjustment of Student Ratings of Teachers for Extrinsic Influences

Abstract

A frequently stated objection to the use student ratings to measure teaching performance is that they reflect extrinsic influences-class conditions over which the instructor has no control. This article presents a method adjusting for these influences using multiple regression analysis, and provides an illustration its application. The evidence on the importance extrinsic influences is mixed. Gage reports evidence from one study that teachers of the lower level consistently received less favorable mean ratings than did those more advanced courses. Teachers in with 30-39 students consistently received lower ratings than did those in with more or fewer students. Teachers on-campus received significantly worse ratings than did those off-campus courses. Finally, teachers elective received consistently more favorable ratings than did instructors required courses [1, pp. 58-59]. On the other hand, Remmers and Elliot report that, while graduate students rated instructors higher than undergraduates, among undergraduates the student's year college was not correlated with the instructor's rating [2]. Guthrie finds that class size was not correlated with student ratings teaching at Washington [3]. Again Costin, Greenough and Menges report that required courses, upper-class students, and majors correlate with student ratings instructors [4]. But Mirus finds that required and fourth-year (as compared with lower-level) at the University Alberta do not significantly affect the instructor's rating, and class size has very little (albeit, positive) effect on the rating [5]. In short, the mix evidence establishes doubt about the reliability a raw rating as a measure teaching performance. Therefore, in assessing the performance 36 instructors teaching 164 classes from 1971 to 1974 in the Department Economics, University Hawaii, we sought to eliminate the influence extrinsic class characteristics, and to obtain adjusted ratings that reflect the remaining intrinsic characteristics over which the instructor does have control. Our method employs multiple regression analysis, in which Y = the mean instructor ratings on a five-point scale given by students in a class Xia = class size Xlb = log class size Xlc = reciprocal class size