Making Better Use of Student Evaluations of Teachers

Abstract

shall we conclude from these charges? I conclude is this: we feel more keenly the problems of evaluation when we are on the receiving end than when we're on the giving end. That is, the charges I've just summarized simply throw a clearer light on the problems in #//evaluation, particularly conventional faculty grading of students. Let me set the problems of student evaluations into a larger context by briefly standing back and talking about evaluation in general. Trustworthy, fair evaluation means giving God's ver dict?finding the single verdict that all right-minded, good readers would agree on. The problem is that God isn't telling her verdict, and we cannot get readers to agree?not even good readers. It may sound extreme to invoke God here, but we can't be cavalier about evalu ation in education. A single student's evaluation of a teacher doesn't carry much weight, but a single teach er's grade for a student often carries a lot?for example, in an application for a scholarship or a job or profes sional school. We can't just take a fashionably theoreti cal view of the grades we give: Oh well, of course my grades are 'situated' and 'interested'?so what else is new? Because grades carry heavy consequences, we cannot take anything less than genuine fairness as our goal?God's view, correctness?yet we know that trust worthy, fair evaluation is not possible. I'm not saying anything new. We've long seen this predicament on many fronts. Research in eval uation has repeatedly shown that if we give a paper to a set of read ers, they tend to give it the full range of grades (see Diederich for a classic exploration; for an indication of how long people have noticed this problem, see Starch and Elliott [three cita tions] and the summary of their work in Kirschenbaum, Sidney, and Napier 258-59). We know the same thing from literary criticism and theory. The best critics disagree about the quality of texts?even about what texts mean?and nothing in literary or philosophical theory gives us any agreed-on rules for settling such disputes. Barbara Herrnstein Smith may not be too cynical in concluding that when ever we have interreader reliability, we have something fishy. And students know the same thing from their controlled experiments of handing the same paper to different teachers and getting different grades. (Perhaps this explains why we tend to hate it when students ask their favorite question, What do you want for an A?: it rubs our noses in the unreliability of our grades.) Champions of holistic scoring will reply that they do get readers to agree, but they get that agreement by training the readers before and during the scoring sessions?that is, by getting them to stop using the conflicting criteria and standards they normally use outside the scoring sessions. Thus the reliability in holistic scoring measures not how texts are valued in natural settings by actual readers but only how they are valued in artificial settings with imposed agreements. Nevertheless, these practical and theoretical problems don't permit us to decide we can get along without eval uation. Everyone seems to agree that we need some kind of evaluation of students by teachers. And even cynical teachers realize that we can't just refuse to have our students evaluate us. Those who ignore the other reasons still must acknowledge that colleges operate in a