Development of a student engagement score for online undergraduate engineering courses using learning management system interaction data

Abstract

AbstractAlthough researchers agree that student engagement in online courses is a function of time dedicated to course‐related activities, there is little consensus about the best way to quantify the construct. This study introduces a measure for undergraduate engineering students' engagement in online courses using their interactions with their online course learning management system (LMS). Data from 81 courses offered by three fully online, undergraduate engineering degree programs generated a total of 3848 unique student–course combinations (approximately 2.7 million rows of LMS interaction data), to which we applied a five‐step process to calculate a single score representing student LMS engagement. First, we converted the students' LMS interaction data into a set of natural features representing the time they spent per 3‐day period on various course elements, such as quizzes, assignments, discussion forums, and so forth, and how these times changed across the duration of the course. We then used the natural features to derive 216 relative features describing deviations from typical interaction patterns among students in the same course. Next, we conducted association rule mining on a training portion of the data set to generate rules separately describing the behavior of students who completed the course (completers) and those who chose to drop early (leavers). The rules generated were applied to students from the testing portion of the data set to compute the percentage of unique rules met by completers and leavers. Finally, the mathematical difference between the percentages of completer and leaver rules met by each student was found to be the best measure of student engagement.