A Comparison of Split-Half and Multilevel Methods to Assess the Reliability of Progress Monitoring Outcomes

Abstract

One consideration for selecting progress monitoring tools is the reliability in which the instrument measures student response to instruction. Researchers and vendors establish reliability of growth using two analytic methods: (a) calculating slopes to even and odd observations for each student and correlating the resulting slopes (split-half), and (b) estimating a multilevel model and computing the ratio between true and observed variance in growth estimates. It is unclear whether reliability estimates from either analytic method systematically differ. Therefore, it is also unclear whether recommendations regarding the requisite length of time to collect data to achieve sufficient levels of reliability would vary based on the analytic method used to calculate coefficients. Results from this study indicate that the multilevel method yielded systematically higher reliability estimates compared with the split-half method. Bootstrapping confirmed that differences were statistically significant. Researchers and practitioners should be aware that split-half and multilevel reliability estimates are not interchangeable and recommendations regarding the frequency and duration educators should follow when collecting data may depend on the manner in which reliability is estimated.