Words of Wisdom: A Journey through the Realm of NLP for Learning Analytics - A Systematic Literature Review

Abstract

Learning Analytics (LA) is one of the world’s most influential research fields related to educational technology. Among many themes that the LA community considers, the application of Natural Language Processing (NLP) algorithms has been largely adopted to extract information from textual data generated in learning environments (e.g., student essays and short answers, online discussion and chat). NLP can shed light on the learning process and student outcomes in different contexts. Based on the importance of NLP for education, this paper conducted a systematic literature review of the application of NLP to understand how the LA community has been applying this method. Our methodology includes automatic and manual methods to extract information about authors, relevant papers, and specific data related to educational applications and algorithms used in the field. This review selected 156 papers that reveal essential aspects of the topic, such as: (i) the majority of the works focused on the analysis of online discussions and essay assessment; (ii) in general, the authors did not apply the developed models in real settings; (iii) recent papers selected start to evaluate deep learning models (e.g., BERT) more frequently; (iv) the datasets used in the experimentation are usually small and containing English text; (v) the average models performance reaches 0.54 and 0.79 of Cohen’s Kappa and Accuracy, respectively. The results of this study and its practical implications are further discussed.