Learner analytics: Hindsight evaluation at course-level

Abstract

The concept of student engagement is a contentious construct. The task of learner analytics (LA) to meaningfully measure student engagement is therefore complicated both by a lack of agreement over what is being measured and a discomfort or lack of confidence around what collated data might believably indicate. This challenge is made harder by availability, accuracy and reliability of data feeds. The aim of LA would be to collate and share early measures of engagement that can be used predictively to support learners’ experience and outcomes. However, most HEI LA are descriptive and therefore of limited utility. Where the LA available are descriptive, this paper explores how credible such LA might be when used at course level. This study supports an analysis of comprehensive data gathered within and beyond LA for a level 4 cohort in one programme across the 2019-20 academic year. It also draws on data relating to study completion, with the benefit of hindsight giving further insights to the utility of LA data available earlier in students’ journeys. Given the actual outcomes for these 2019 starters, the study cohort’s understanding of ‘engagement’ is then applied to support insights to their own measurable indicators of interaction and actions that might best enable constructive engagement. Meaningful correlations were noted between use of E-resources and student outcomes and the most significant indicators of risk were found to be extensions, fails and non-submissions for assignments in the first semester of level 4 and average grades <39% by the end of level 4. Study recommendations include supporting better and more confident access to literature content and targeting timely interventions at students flagged by the most significant indicators of risk.