Trusting Teachers’ Judgments: A Validity Study of a Curriculum-Embedded Performance Assessment in Kindergarten to Grade 3

Abstract

Teacher judgments of student learning are a key element in performance assessment. This study examines aspects of the validity of teacher judgments that are based on the Work Sampling System (WSS), a curriculum-embedded, performance assessment for preschool (age 3) to Grade 5. The purpose of the study is to determine if teacher judgments about student learning in kindergarten to third grade are trustworthy if they are informed by a curriculum-embedded performance assessment. A cross-sectional sample composed of 345 K-3 students enrolled in 17 classrooms in an urban school system was studied. Analyses included correlations between WSS and an individually administered psychoeducational battery, four-step hierarchical regressions to examine the variance in students’ spring outcome scores, and receiver-operating-characteristics (ROC) curves to compare the accuracy of WSS in categorizing students in terms of the outcome. Results demonstrate that WSS correlates well with a standardized, individually administered psychoeducational battery; that it is a reliable predictor of achievement ratings in K-3; and that the data obtained from WSS have significant utility for discriminating accurately between children who are at risk (e.g., Title I) and those who are not at risk. Further discussion concerns the role of teacher judgment in assessing student learning and achievement.