Detecting patterns of engagement in a digital cognitive skills training game

Abstract

Indicators of behavioral engagement derived from log data may provide insight about variation in participants' interactions with and the efficacy of digital cognitive skills training games. We first sought to determine whether distinct groups of adolescents (N = 163; Mean age = 14.1 years, SD = 1.3) could be identified based on variables derived from digital log data collected while participants played a game designed to enhance inhibitory control. We then examined whether these data-driven participant groupings were associated with improvement in inhibitory control. Latent class mixture modeling was conducted both with reaction time and a measure of response accuracy (d’) of log data. Results indicated two distinct classes based on reaction time, and four classes based on response accuracy over the course of training. Class membership based on reaction time was associated with differential improvements in performance on a subsequent standardized measure of inhibitory control. The findings point towards the need for formative metrics of progress, as well as the need for more adaptive and user-centered cognitive skills interventions. Our findings suggest that there may be some utility in analyzing log data as an indicator of student engagement, particularly when used in complement with more traditional measures of performance.