Performance increases in mathematics within an intelligent tutoring system during COVID-19 related school closures: a large-scale longitudinal evaluation

Abstract

The closure of schools due to COVID-19 disrupted learning routines of thousands of students, resulting in reported performance decreases, especially among lower-performing students. However, some studies on students’ performance within intelligent tutoring systems (ITS) also found significant performance increases for times of school closures as compared to before. However, little is known about students’ longitudinal performance trajectories within ITS. Accordingly, we evaluated longitudinal data from German students (n ≈ 2,700 students; n ≈ 5 million problems) enrolled in an ITS for learning mathematics from January 2017 until the end of May 2021 to investigate the effect of periods of school closures (first and second) on students’ performance within the ITS during, between, and after school closures. We observed significant performance increases for both lower- and higher-performing students during, between, and after COVID-19 related school closures. Importantly, these improvements were more pronounced for lower-performing students compared to higher-performing students. Together, these results suggest that ITS may have helped to maintain mathematics learning, particularly for lower-performing students during COVID-19 related school closures and that these beneficial effects persisted at least for the following months when schools opened again. As such, the use of ITS for learning mathematics seems an appropriate approach for distance learning during times of crisis.