Making Learning Visible: Student and Peer Evaluation

Abstract

I'll begin this response to Mark Giese's final journal entry by taking up his discussion of student assessment, focusing on the issue of subjectivity in grading. I'll also take Mark's lead and offer a few additional reflections on the process of peer review of teaching and return, by the end of this piece, to the concept of the scholarship of teaching because it crystallizes the focus on student-and instructor-learning. The knowledge and skills that Mark and Nancy emphasized in their course created some assessment challenges. Traditional evaluation techniques, like multiple choice tests, didn't seem adequate to the task of measuring complex performances. Accordingly, Mark and Nancy used a variety of approaches to gauge student learning. Mark writes about the subjectivity inherent in the grading process, but argues that despite the perils, evaluation provides students with feedback that they can use to improve their learning. This term, I am teaching a graduate-level course on qualitative research methods in education, so the tension between objectivity and subjectivity is on my mind quite a bit. Qualitative researchers typically argue that subjectivity is unavoidable in the research process because the choices we make in research are not neutral and value-free. Since complete objectivity is impossible, a reasonable course of action is for researchers to heighten their consciousness of their subjectivities and how they might affect their research. If you can't eliminate subjectivity, many qualitative researchers would argue, you must find a way to tame it so that it doesn't interfere with the ability to understand and report what is happening in the research setting (for a discussion, see Maxwell, 1996; Peshkin, 1991). Instructors, I think, need to take a similar approach to dealing with subjectivity in student assessment because it is clear that many important learning outcomes can't be assessed using what we have come to think of as objective measures. For example, most of us seek to promote students' abilities to use their knowledge and skills to solve the messy problems they encounter in the world. We are fond of saying that real-world problems don't have a single right answer, so students must learn to grapple with incomplete information and conflicting evidence, weigh the advantages and disadvantages of a variety of solutions, evaluate alternatives, and come to justifiable conclusions. A good example of how objectivity and subjectivity meet in assessment comes from the field of engineering where students often compete to produce the best' design solution to a given problem, such as how to build a more efficient and environmentally friendly car. Students must correctly apply mathematical and scientific knowledge and skills in addressing such a need, but in evaluating the final designs, faculty and other expert judges, often industry representatives, also consider other criteria to render a final decision. These might include cost-effectiveness, originality, creativity, energy efficiency, aesthetic appeal, aerodynamics, marketability, and so on. Subjective judgments are part of a holistic appraisal of success. Rather than apologize for the intrusion of subjective factors in grading, instructors would be better served by acknowledging the important role of expert judgment in assessments of student learning.1 Engineers assume that experts know what expert performance looks like and can articulate its features. Similarly, in their respective fields of study, faculty members are expert judges who know what criteria should be used to evaluate a particular assignment. They also know what differentiates a poor performance from an excellent one. In my experience, many instructors are uncomfortable with authentic assessment of student learning because they have not thought carefully about the criteria and standards of performance that they will use to evaluate a given performance. …