Use of Natural Language Processing (NLP) Tools to Assess Digital Literacy Skills

Abstract

The framework for Education 4.0 highlights the need for new assessment models of digital competencies to be incorporated into the curricula of Higher Education Institutions. Additionally, including new artificial intelligence tools (AI) can significantly help assess student skills development more flexibly and effectively. This study versatilely implemented natural language processing (NLP) tools to strengthen the active learning approach to developing adequate educational solutions. The NLP tools proved ideal for evaluating soft skills in engineering. The study sample comprised 286 students enrolled in sustainability and energy efficiency subjects during six consecutive semesters from 2019 to 2021. The study's mixed methodology used a four-group Solomon-type design with two types of validated instruments used as pre-Tests and post-Tests: a) qualitative (surveys, questionnaires, interviews, and rubrics) and b) quantitative (tools for statistical data management). The study results confirmed that NLP tools are helpful to reduce teachers' training biases in the evaluations of students' digital literacy skills. In addition, the results showed that NLP tools can complement the evidence that instructors share with students in review and feedback sessions and improve their perception of the validity and reliability of the assessment instruments.