Assessment of Higher-order Thinking Skills in Conversation Situation

Abstract

Assessment of students’ higher-order thinking skills is a key issue in education, and data-driven multimodal learning analytics is the way forward. Mining the essential content of collaborative conversations is the key to the assessment of higher-order thinking skills, and the integration of learning situations is crucial. In this paper, we provide digital collaborative learning situations through a virtual simulation platform. Based on the conversation data, decision-making behaviour data and environmental data generated in the collaborative learning process, a set of data-driven higher-order thinking skills identification methods integrating situation and conversation are designed. The method includes four steps: situation identification and modelling, text extraction, situational semantic analysis, and semantic flow-based modelling. Compared with traditional methods such as questionnaires and interviews, the ability identification driven by conversation data realizes the direct representation of higher-order thinking skills, which has real-time performance and higher reliability. What’s more, integrating situational information into the learning analysis model improves the accuracy of ability identification results from the pragmatic level.