Explaining reading variance by student subgroup: should we move beyond oral reading fluency?

Abstract

BackgroundStudies have demonstrated strong predictive and concurrent validity of curriculum‐based measures (CBMs) with high‐stakes reading tests, mainly using oral reading fluency (ORF) to predict outcomes. Some studies have found that CBMs' predictive accuracy might vary across demographic subgroups of students. In this study, we investigate whether additional CBMs of reading comprehension account for variance beyond ORF and whether these reading variables predict reading comprehension differently for demographic subgroups.MethodsCurriculum‐based measures were administered to 9,967 students at the end of Grade 3 in the United States, concurrent with a high‐stakes standardised state reading comprehension achievement test (RCAT). Hierarchical regression models with RCAT as the outcome were used to test (a) whether CBMs of reading comprehension accounted for significant variance beyond ORF and (b) if the reading variables differentially predicted RCAT for different subgroups of students.ResultsOral reading fluency explained significant variation in RCAT, and the addition of comprehension CBMs accounted for small amounts of unique variance. The contribution of ORF in explaining variation in reading comprehension varied by demographic subgroups.ConclusionsThe contributions of comprehension CBMs in predicting high‐stakes standardised reading comprehension tests were not meaningful, only accounting for small amounts of unique variance. Additionally, the contribution of ORF in explaining variation in the high‐stakes standardised reading comprehension test varied by student demographics, likely due to complex reasons specific to those individual factors.HighlightsWhat is already known about this topic CBMs of ORF account for significant variance in reading comprehension achievement on high‐stakes standardised reading tests. Failure to include some measure of comprehension in CBMs could be problematic because fluency does not ensure comprehension. When comprehension CBMs have been included, findings have been mixed. Some studies have pointed to differences in how well ORF predicts reading comprehension for various subgroups of students. What this paper adds We examine the contribution of comprehension CBMs over and above ORF in predicting student performance on a high‐stakes reading comprehension test using a large, diverse sample in the United States. We replicate others' findings on the contribution of ORF in predicting variance in students' high‐stakes reading comprehension test scores. We find that the contributions of ORF differ by student subgroups, specifically socioeconomic status, limited English proficiency, and race/ethnicity. Implications for theory, policy or practice Conducting and analysing assessments takes a significant amount of teachers' time. Our findings suggest a lack of utility in using additional CBMs beyond a measure of ORF. Given the consistency of our findings with prior studies, teachers should use ORF as a screening measure and then shift to diagnostic measures to better understand students' performance. Because ORF differentially predicts performance on a high‐stakes reading comprehension test based on subgroups of students, alternative assessments should be considered for particular groups of students.