A review on multimodal online educational engagement detection system using facial expression, eye movement and speech recognition

Abstract

During this lockdown period, an online educational engagement system plays a vital role to enrich the knowledge of learners in various fields without interrupting their learning process. Online educational engagement systems include all the activities of a learner like listening, reading, writing, and so on. While participating in these activities, a participant may show various levels of engagements like fully engaged, partially engaged and completely not engaged. The participation of online learners has to be identified for an effective learning process. The existing literature could be classified depending upon the learners’ participation as automatic, semi-automatic and manual. Further it could be sub categorised based on the data types used to identify the engagement system. In this paper, a review on computer based automatic online educational engagement detection systems is presented. Several educational engagement methods are applied for computer based online engagement detection systems. In these systems examining a participant’s presence and attention with the modalities of facial expression, eye movement and speech are found to be a challenging task. In this work, it is also identified that there are few challenges like preparation and usage of proper datasets, identifying suitable performance metrics for different tasks involved and providing recommendations for future enhancement of online educational engagement detection by combining the modalities of facial expression, eye movement and speech are still unattended. Though there are several research gaps involved, an online educational engagement system will help the learners to engage themselves in a productive way of learning and getting evaluated efficiently and effectively during the lockdown period of pandemic disease COVID-19 without interrupting their learning process and gaining knowledge. © 2021 Karadeniz Technical University. All rights reserved.