Understanding variation in classroom quality within early childhood centers: Evidence from Colorado's quality rating and improvement system

Abstract

This study examines variability in quality across classrooms within early childhood centers and its implications for how quality rating systems (QRSs) capture center-level quality. We used data collected for administrative purposes by Qualistar Colorado which includes the environmental rating scale (ERS) collected in all classrooms in the 433 centers participating in Colorado's QRS between 2008 and 2010. We conducted variance components analysis for the ERS and found that between 26% and 28% of the variation in quality captured by the ERS occurred across classrooms within the same center serving children in the same age range. This finding reveals that capturing center-level quality based on average ERS will often miss important within-center quality differences and points to the merits of using “no score below” rules along with rating tier cutpoints in determining center-level ERS. Most QRSs assess center-level quality for a randomly selected subset of classrooms. To test the implications of cross-classroom quality variation for this practice, we simulated four classroom selection strategies in current use: selecting 50% of the rooms, 33% of the rooms, two rooms, or one room. In general, the larger the share of classrooms measured under a selection rule, the lower the chance that a center's rating tier will be misclassified. The error rates under each selection rule also depend on the extent of cross-classroom quality variability, how centers are distributed by size, and the QRS structure. QRS designers, therefore, need to consider the tradeoff between the costs of measuring more classrooms in each center versus the costs of misclassifying centers. The paper quantifies the magnitude of these tradeoffs using the Colorado data and two illustrative QRSs. The implications of our findings for QRS designers, parents, and other stakeholders are discussed.