An Analysis of Learning Rate and Curricular Scope: Caution When Choosing Academic Interventions Based on Aggregated Outcomes

Abstract

.We aggregated and analyzed single-case research targeting math-fact fluency to investigate whether learning (behavior change) data were sufficient to summarize and compare intervention outcomes or if learning rate (learning/cumulative instructional time) data would provide divergent effect size (ES) results. We also extracted the curricular scope or set size data (how many items were targeted) and evaluated the relationship with obtained ESs. The results suggest that ESs calculated without accounting for cumulative instructional time and curricular scope result in different estimates of effect than ESs that do account for cumulative instructional time and curricular scope. Discussion focuses on the importance of using learning rate and curricular scope when comparing intervention effects and limitations associated with drawing relative-effectiveness conclusions by aggregating outcomes across studies.