Improving learning science: Evaluating and comparing academic interventions using measures of learning speed

Abstract

AbstractMeasures of learning speed (i.e., the amount of learning/cumulative time learner spends engaged in an intervention) are rarely included in research designed to evaluate and compare academic interventions. We build a case that includes analyses of learning speed metrics in academic intervention research can provide more useful information for prevention and remediation efforts, deter researchers from drawing misleading data‐based conclusions, and promote scientific parsimony. Research is reviewed where the same learning data is analyzed across different metrics (i.e., same amount of learning, different measurement scales) including learning per event, learning speed, and growth. Patterns are analyzed and examples are provided that demonstrate how altering metrics can influence relative effectiveness conclusions and applied recommendations when researchers compare two or more interventions, conduct component analysis or optimization studies, and apply meta‐analytic techniques. Opportunity costs for learners, educators, and learning scientists that may be caused by failure to measure learning speed are identified and analyzed. Our review and analyses support the conclusion that, in most instances, researchers can better inform prevention and remediation efforts by including precise measures of learning speed as dependent variables when they evaluate and compare interventions designed to enhance academic skills. Limitations and challenges associated with measuring learning speed are described and directions for future research are provided.