Student engagement study based on multi-cue detection and recognition in an intelligent learning environment

Abstract

Student engagement has great impact on learning performance. It’s necessary to investigate student engagement objectively from learning behavior. In this paper, we propose a student engagement study approach in an intelligent learning environment, which automatically detects and analyses multiple learning behavioral cues based on five modules, i.e., attendance management, teacher-student (T&S) communication, visual focus of attention (VFOA) recognition, smile detection and engagement analysis. Attendance management matches the student’s identity and locates his/her profile using face recognition. T&S communication provides an additional channel of Question and Answer (Q&A) between a teacher and students for students’ behavioral engagement analysis via their cell phones. VFOA recognition is used to recognize students’ cognitive engagement through capturing students’ attention based on the estimated head poses, visual environment cues and prior states in class. Smile detection achieves students’ affective engagement through spontaneous smile expression classification. Finally, a tree-structural engagement model is proposed to decide student engagement based on multi-cues of one’s behavioral, cognitive and affective engagement. We thoroughly evaluated each module for engagement study on some public available datasets and practical video sequences in class applications. The experimental results suggest that the proposed approach can automatically detect and analyze student class engagement objectively and effectively.