Emotion estimation model for cognitive state analysis of learners in online education using deep learning

Abstract

AbstractFor Facial Expression Recognition, occlusion and position change that may drastically alter facial expressions are two important challenges (FER). Due to advances in automated FER over the last several decades, it has received less attention in the real world, where occlusion‐ and pose‐invariant aspects of FER are critical. Online education learners' cognitive states may be assessed using this paper's focus on real‐world stance and occlusion robust FER. The human visual system's attention mechanism inspired us to develop a new kind of spatial attention network (SAN‐CNN). Saliency characteristics and spatial importance between adjacent pixels are emphasized in the SAN‐CNN model. Preprocessing an input picture using a median contour filter is used here initially, followed by mask‐based ROI for segmentation. Using CNNs for facial recognition, landmark localization, and head position estimation based on spatial attention networks, it is possible to do emotional categorization. Using the Kaggle public video‐based facial expression datasets, we were able to demonstrate that our proposed approach is more accurate and faster than the usual techniques. In addition, we evaluated the suggested method's performance metrics with those of the already used approaches. FER with occlusion and variant posture performs better than standard approaches using our suggested method, as shown by the results of the experiments.