Is the Child Really What’s Being Rated? Sources of Variance in Teacher Ratings of Socioemotional Skills

Abstract

This study examined three potential sources of variance in teacher ratings of the socioemotional skills of children: occasion, child, and teacher. The ratings were prepared using the Devereux Student Strengths Assessment. In total, 344 children were assessed in both the fall and spring of kindergarten and first grade, providing a total of 1,067 ratings across 106 teachers. We employed cross-classified random effects models to account for multiple occasions of measurement that were nested within both children and teachers, which allowed us to determine the amounts of variance attributable to occasion, children, and teachers. Overall, occasion accounted for the greatest proportion of variance (43%-50%) across a variety of socioemotional subscales, followed by child (36%-46%), and teacher (11%-16%). Findings suggest that caution is warranted when using these scales for high-stake decisions.