Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievement

Abstract

In the MOOC forum discussions, emotional and cognitive engagement are two prominent aspects of learning engagement. Moreover, emotional and cognitive engagement have an interactive relationship and can jointly predict learning achievement. However, these interwoven relationships have not been thoroughly explored. Furthermore, the limitations on detection methods for emotional and cognitive engagement have hindered the practice and theory progress. This study aimed to develop a novel text classification model to automatically detect emotional and cognitive engagement and investigate their complex relationships with achievement, which are beneficial for improving learning engagement and historically low completion rates of MOOCs. Firstly, this study proposed a robust and interpretable NLP model called the bidirectional encoder representation from the transformers-convolutional neural network (BERT-CNN). Compared with models in previous studies, it improved the F1 values of emotional and cognitive engagement recognition tasks by 10% and 8%, respectively. Secondly, this study used BERT-CNN to analyze 8867 learners’ discussions in a MOOC forum. Structural equation modeling indicated that emotional and cognitive engagement have an interactive relationship and a combined effect on learning achievement. Specifically, positive and confused emotions contributed more to higher-level cognition than negative emotions. Co-occurring emotion and cognition indicators jointly predicted learning achievement with higher reliability. In summary, this study has significant methodological implications for the automated measurement of emotional and cognitive engagement. Moreover, the study revealed the dominant role of emotional engagement on cognitive engagement and provided suggestions for improving MOOC learners' achievement.