Do learning rates differ by race/ethnicity over kindergarten? Reconciling results across gain score, first-difference, and random effects models

Abstract

The question of whether students’ school-year learning rates differ by race/ethnicity is important for monitoring educational inequality. Researchers applying different modeling strategies to the same data (the ECLS-K:99) have reached contrasting conclusions on this question. We outline the similarities and differences across three common approaches to estimating gains and heterogeneity in gains: 1) a gain score model (with intercept), 2) a first-difference (FD) model (in some cases equivalent to regression-through-the-origin [RTO] and student fixed effects models), and 3) a student random effects (RE) model. We show via simulation that FD/RTO and RE models produce estimates of learning rates – and group differences in learning rates – with more favorable RMSD compared to the gain score model with intercept. Using data from the ECLS-K:99, we demonstrate that these precision differences lead to contrasting inferences regarding learning rate heterogeneity, and likely explain the inconsistencies across previous studies.