Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques

Abstract

The onset of digital education, propelled by the global COVID-19 crisis, has revolutionized the education sector, presenting unique difficulties, including the crucial task of preserving academic honesty. This study explores the possibilities of computer vision technologies, specifically face recognition and detection systems, to deter dishonest practices in online learning contexts. In this article we aim to construct efficacious strategies that leverage these technologies to track student actions in real-time and alert educators about possible cheating instances. This study presents two innovative models addressing cheating in online learning settings using cutting-edge computer vision techniques. Our initial model is an ensemble learning based face recognition system that blends the functionalities of three different deep learning (DL) structures: VGG, MobileNet, and DenseNet. This ensemble learning approach aims to offset the shortcomings of individual models while amplifying the overall effectiveness. The model’s efficiency will be gauged by juxtaposing it with other models and testing its performance against renowned benchmark datasets. Following this, we propose a second model designed for real-time face and cheating detection. This model integrates the FaceMesh model, facial landmarks analysis, and head pose estimation to identify possible cheating behaviors, such as significant shifts from a neutral or forward-facing head position. This model’s efficiency will be assessed through testing in simulated cheating scenarios and using authentic data from online learning contexts. Upon testing and validation, our proposed models have shown encouraging outcomes. The ensemble learning model outstripped individual models by attaining a remarkable accuracy rate of 91% through soft voting. Furthermore, the face detection system showcased sturdy abilities in recognizing faces under diverse conditions and accurately pinpointed potential cheating behaviors based on head pose estimation.