Detecting Learner Drowsiness Based on Facial Expressions and Head Movements in Online Courses

Abstract

Drowsiness is a major factor that hinders learning. To improve learning efficiency, it is important to understand students' physical status such as wakefulness during online coursework. In this study, we have proposed a drowsiness estimation method based on learners' head and facial movements while viewing video lectures. To examine the effectiveness of head and facial movements in drowsiness estimation, we collected learner video data recorded during e-learning and applied a deep learning approach under the following conditions: (a) using only facial movement data, (b) using only head movement data, and (c) using both facial and head movement data. We achieved an average F1-macro score of 0.74 in personalized models for detecting learner drowsiness using both facial and head movement data.