The different relationships between engagement and outcomes across participant subgroups in Massive Open Online Courses

Abstract

Previous research has found that early engagement in MOOCs (e.g., watching lectures, contributing to discussion forums, and submitting assignments) can be used to predict course completion and course grade, which may help instructors and administrators to identify at-risk participants and to target interventions. However, most of these analyses have only focused on the average relationships between engagement and achievement, which may mask important heterogeneity among participant subgroups in MOOCs. This study examines how the relationship between engagement and achievement may vary across the four common behaviorally identified participant subgroups (“disengagers,” “auditors,” “quiz-takers,” and “all-rounders”) in three MOOC courses offered on the Coursera platform. For each of these subgroups, we used measures of behavioral and cognitive engagement from the first half of the ten-week courses to predict two outcomes: course grade and overall lecture coverage. Results indicate that the same engagement measure may be oppositely associated with achievement for different subgroups and that some engagement measures predict achievement for one subgroup but not another. These findings provide insight into both the benefits and the complexity of studying patterns of engagement from behavioral data and provide suggestions on the improvement of identification of at-risk participants in MOOCs.