Integrated infrared imaging techniques and multi-model information via convolution neural network for learning engagement evaluation

Abstract

Infrared (IR) imaging sensor can achieve the clean images in the online learning environment. Thus, it can be utilized to analyze the facial expressions for learning engagement evaluation task. In this paper, we proposed an integrated evaluation model to detect students' online learning engagement by using infrared images with facial expressions and mouse movement data. The integrated model consisted of two sub-models which used convolution neural network (VGG-16), one was utilized to evaluate learning engagement level of IR images captured by infrared imaging equipment during learning process, the other one was used to evaluate engagement level of mouse flow. The two results were fused with the log data, and the final learning engagement level was obtained after another convolution neural network processing. Finally, we compared the results calculated by the integrated model and online student engagement scale (OSES). Experimental results demonstrate that there is a strong correlation between the two results, which showed that the model is feasible and effective for learning engagement evaluation. The infrared image model information can effectively extract the facial feature and use for learning engagement evaluation task.