“This Student Needs to Stay Back”: To What Degree Would Instructors Rely on the Recommendation of Learning Analytics?

Abstract

Learning analytics (LA) systems are becoming a new source of advice for instructors. Using LA provides new insights into learning behaviours and occurring problems about learners. Educational platforms collect a wide range of data while learners use them, for example, time spent on the platform, passed exams, and completed tasks and provide recommendations in terms of predicted learning success based on LA. In turn, LA might increase efficiency and objectivity in the grading process. In this paper, we examine how instructors react to the platform’s automatic recommendations and to which extent they consider them when judging learners. Drawing on an adaptive choice-based experimental research design and a sample of 372 instructors, we analyze whether and to what degree instructors are influenced by the provided data and recommendations of an unknown LA system. In a follow-up study with 95 teachers, we describe the differences in the use of data between learners and the influence of early warning systems. All in all, we show the influence of automatic evaluation on teachers.