The effect of adaptivity in digital learning technologies. Modelling learning efficiency using data from an educational game

Abstract

AbstractDuring the last decade, many governments and ed‐tech companies have demonstrated an increased interest in digital personalised learning, which resulted in a variety of often game‐like adaptive learning environments. However, there has been limited attention for the impact of these personalised learning technologies on children's learning efficiency. Does digital personalised learning, like popular claims insist, foster learning in young children? This study attempts to empirically validate the beneficial impact of adaptive learning technology by analysing log‐data from the Number Sense Game (NSG), an educational game that trains early numerical skills. In total, 81 children were randomly assigned to use either an adaptive or a non‐adaptive version of the NSG in six sessions in a three‐week period. Using a longitudinal item response model children's progress within and across sessions was modelled and compared between the two versions of the game. Regardless of the version of the NSG, children demonstrated progress within and across sessions. However, compared to the non‐adaptive NSG, the progress across sessions was stronger in the adaptive NSG. These results provide empirical evidence that adaptive learning environments can improve learning efficiency in young children. Practitioner notesWhat is already known about this topic Adaptive digital technologies are able to address students’ individual needs. Pretest posttest intervention research on the effectiveness of adaptive digital technologies revealed mixed results. What this paper adds Learning effectiveness is different from learning efficiency. Learning efficiency can be operationalised using log‐data to model students’ progress over time. Adaptive digital technologies increased students’ learning efficiency compared to non‐adaptive digital technologies. Implications for practice and/or policy Log‐data can be obtained unintrusively and provides fine‐grained information about students’ learning. Researchers are encouraged to investigate the effects of adaptive digital technologies in terms of learning effectiveness and learning efficiency.