Evaluating the Criterion Validity and Classification Accuracy of Universal Screening Measures in Reading

Abstract

Educators use universal screening to identify students who may be at risk for not meeting proficiency on the state assessment. Given the potential high-stakes of state tests, using accurate screeners is critical. Independent research is emerging on screeners such as the Measures of Academic Progress (MAP), a computer adaptive test, and the Strategic Teaching Evaluation of Progress (STEP), a developmental reading assessment. This study evaluated the criterion-related validity of MAP and STEP with a state assessment. Additionally, the utility of each screener to distinguish between students at risk and not at risk was evaluated using multiple screening approaches. Participants were two cohorts (Cohort 1 N = 255; Cohort 2 N = 122) of children enrolled in two public charter schools. MAP and STEP were administered in spring of second grade and fall of third grade. Results suggested MAP and STEP scores were significant strong predictors of third-grade state assessment scores. STEP scores explained a small amount of variance in state assessment scores beyond MAP scores alone. Findings support using MAP and STEP to identify students who may be at risk; however, MAP is sufficient as a single screener, using locally derived cut-scores or cut-scores linked to the state assessment.