Four methods of identifying change in the context of a multiple component reading intervention for struggling middle school readers

Abstract

The results from controlled intervention research have indicated that effective reading interventions exist for children with reading difficulties. Effect sizes for older struggling readers, however, typically have not matched the large effects demonstrated with younger children. Standardized effect sizes for intervention/control comparisons obscure important individual differences within intervention and control groups—differences potentially relevant to the who and why of intervention success. The present study reports the outcomes of PHAST Reading, a research-based multiple component reading intervention. Participants were 270 Grade 6, 7, and 8 students reading significantly below age-level expectations, who participated in a year-long intensive small-group intervention. Four methods were applied to characterize individual change: (a) normalization relative to age-appropriate standards; (b) statistically-reliable pre–post change using the Jacobson–Truax index; (c) individually-estimated growth rates using hierarchical linear modeling; and (d) change to a fixed criterion across multiple measures. Each method was evaluated for its ability to identify intervention outcomes, replicate traditional group-based effect size metrics, and characterize individual differences across participants depending on whether change was demonstrated. Each method replicated traditional group-based effect sizes, with advantages in consistency and predictive power for the reliable change index and growth curve approaches.