S3DCN-OLSR: A shallow 3D CNN method for online learning state recognition

Abstract

The repeated recurrence of COVID-19 has significantly disrupted learning for students in face-to-face instructional settings. While moving from offline to online instruction has proven to be one of the best solutions, classroom management and capturing students' learning states have emerged as important challenges with the increasing popularity of online instruction. To address these challenges, in this paper we propose an online learning status recognition method based on shallow 3D convolution (S3DC-OLSR) for online students, to identify students' online learning states by analysing their micro-expressions. Specifically, we first use the data augmentation method proposed in this paper to decompose the students' online video file into three features: horizontal component of optical flow, vertical component of optical flow and optical amplitude. Next, the students' online learning status is recognised by feeding the processed data into a shallow 3D convolution neural network. To test the performance of our method, we conduct extensive experiments on the CASME II and SMIC datasets, and the results indicate that our method outperforms the other state-of-the-art methods considered in terms of recognition accuracy, UF1 and UAR, which demonstrates the superiority of our method in identifying students’ online learning states.