Relationship between learning engagement metrics and learning outcomes in online engineering course

Abstract

This research WIP contributes to understanding the relationship between learning engagement in Learning Management System (LMS) and outcomes in an online course. In large engineering courses, it is challenging for instructors to identify who is engaging with course materials at a level necessary to be successful in terms of course outcomes. The purpose of this research WIP study is two-fold: (1) to develop metrics for quantifying learner engagement in online courses, and (2) to explore the relationship between engagement and student success. Our research question is: How does learning engagement relate to course outcomes? We modeled learner engagement on a course level using the following features: number of views per content object, total time spent in the platform, percentage of the course accessed by the learners, percentage of feedback read, and number of attempts per quiz. We used the data collected by the LMS in a large first-year engineering course. We obtained data in Fall 2020, the first semester that many traditional universities were forced mostly or entirely online. After calculating the proposed metrics, we used a linear mixed model to analyze the effect of engagement on learning outcomes. Our linear mixed model shows that all engagement metrics are positively related to the final grade. However, the results also indicate that the relationship between engagement and learning outcomes is not linear; more complex modeling is needed to further explore this relationship.