Evaluating Special Education Teachers’ Classroom Performance: Rater Reliability at the Rubric Item Level

Abstract

Classroom observations are an integral component of teacher evaluation systems, but little is known about who is best qualified to observe and evaluate special educators, who have a specialized skillset, and whether observation instruments adequately reflect their instructional practices. In this study, 19 special education teachers in California and Idaho each contributed three video-recorded classroom lessons. Using rubric items designed to reflect efficacious instructional practices for teaching students with disabilities, school administrators and peers scored the teachers’ lessons. Rater reliability and sources of error variance were examined using generalizability theory. Findings indicate that peers were more reliable raters than school administrators, who did not have expertise in special education, and the school administrators’ ratings varied at the rubric items level. Implications for classroom observation systems are discussed.