Documenting Teaching Excellence within the Peer Review of Teaching

Abstract

The initial autumn email from a communication scholar chairing a department's tenure committee was welcomed and welcoming. I'll be happy to review your candidate's teaching scholarship, I wrote back. I requested artifacts on which I could make an evidence-based peer review - syllabi, course assignments and readings, copies of papers the assistant professor had graded including comments returned to the students, lesson plans, examples of creative computer, visual or audio presentations and tutorials, student assessments - in short, a teaching portfolio. Thanks anyway, the committee chair responded. We can't provide that material. The committee was anxious to reach a decision. The faculty member under review, the committee chair wanted me to know, always received strong student ratings, some of the best in the department. The tenure committee was planning its peer review around student evaluations and the individual's reputation, as well as upon the two classroom visits a committee member conducted to observe the assistant professor at work. My role, apparently, would have been to comment on reputation. I don't know the fate of the faculty member under review, or even the individual's name. The procedures briefly outlined, however, are worthy of discussion. Can the evidence described form the basis of a valid peer review? Let's examine the data that held the tenure committee's interest. What elements of teaching excellence are being rewarded? Which are being left out? Where is the emphasis? Student ratings are frequently used as a basis for peer review (the questionable use of student ratings for salary setting and the problems inherent in using any assessment for both formative and summative purposes are issues for another occasion). Although the research is not unanimous, student reviews carefully tailored to elicit opinions about teaching performance in a specific: instructional environment and administered with focused instructions and a clear explanation of what is at stake, may provide data that is both reliable and valid. Fears that student ratings reward the funniest faculty performers and the easiest graders are unfounded. In fact, students give the highest marks when they feel they have learned the most. The infamous bubble sheet can and often does provide an important data point for those graphing teaching performance. Student ratings are easy to use, but they are also subject to easy abuse. Students are not peers. Their assessments are not based upon the kinds of deep familiarity with, and mastery of, either teaching and learning or the field of communication that resides in the professional educational cohort we think of as peers. Student ratings tend to paint with broad brush strokes. Neither the bubble sheets nor open-ended student questionnaires capture a faculty member's professional pedagogical and curricular judgments, or the course preparation, mentoring, tutoring, grading effort, and assessment reflection that take place outside of the classroom. Student ratings also fail the innovation test. Many of us have had personal experience with at least one botched experiment in which our best attempts at new curriculum or pedagogy turn out well below expectations. Student ratings do not distinguish, however, between time-tested classroom delivery and the development of new pedagogies and curricular enhancements that carry the potential of discovery and the risk of poor classroom reception. Student Testimonials provide an additional and related data point a review committee of peers may consider. Although rarely critical, testimonials afford valuable context when viewed as one tile in a larger assessment mosaic. Unlike student ratings, there is little to suggest they are representative and, as indicators of teaching excellence, they may lack both validity and reliability. Classroom Observation, which a cynical colleague of mine refers to as drive-by assessments, are common. …