Designing an artificial intelligence tool to understand student engagement based on teacher's behaviours and movements in video conferencing

Abstract

Video conferencing is an effective tool that promotes interaction and collaboration, increasing student engagement in online learning. This study is the second phase of design-based research to create a tool to generate a report of engaging teaching videos using deep learning as an artificial intelligence (AI) methodology. In this second phase, the authors have applied the characteristics and indicators of engaging teaching videos identified in the first phase, reported in another study, to develop an Artificial Intelligence enabled tool. Twenty-five recorded lecture videos presented to higher education students were annotated based on the indicators and characteristics of engaging teaching videos. An AI expert has assisted the authors in creating the Artificial Intelligence-enabled tool from the reports generated by this manual annotation. With the assistance of this tool, the engagement enhancing teachers' behaviours and movements can be identified from recorded lecture videos, and a report can be generated on engaging teaching videos. For the classification task of video analysis, the deep learning model is adopted in this research. The model is trained with manually annotated videos and determines class imbalance issues and misleading metrics. The model was further improved by adopting the oversampling technique. The second version of the tool achieved promising outputs with average precision, recall, f1-score, and balanced accuracy of 68, 75, 73, and 79%, respectively, in classifying the annotated videos at the indicator level. The tool can assist the education institutes in creating moderation in the lecture delivery and whether the teachers are utilising the technology effectively. Additionally, this can help teachers recognise the presence or absence of engagement-enhancing behaviours and movements during their video conferencing sessions.