Clustering Learner’s Metacognitive Judgment Accuracy and Bias to Explore Learning with AIEd Systems

Abstract

AbstractMetacognitive monitoring and regulation are key dynamic psychological processes in predicting learning, reasoning, and problem solving across AIEd systems. They are impacted by one’s metacognitive knowledge, skills, and experiences. Understanding the dynamical processes underlying metacognition are critical to design intelligent adaptive scaffolding. Metacognition may be assumed to function at hierarchical levels of abstraction including a local level (i.e., isolated metacognitive judgements) and global level (i.e., self-beliefs). However, metacognitive research for complex learning has traditionally disregarded the concept of a general metacognitive ability due to local level fluctuations in temporal metacognitive accuracy. In our study, we shift our analyses to reflect a global-level approach to study metacognitive judgment ability by measuring both accuracy and bias across a series of metacognitive judgments. Using hierarchical clustering on undergraduates’ (n = 58) metacognitive judgments’ accuracy and bias while learning about nine human biological systems with MetaTutor-IVH, a multimedia-based learning environment, we show that some learners show patterns of global-level metacognitive ability. Specifically, we find that learners who tended to have low metacognitive accuracy across all judgment types performed worse overall on learning outcomes. Examining the bias of these learners, we found they tended to be under-confident across all judgment types. Our work suggests that considering multiple metrics of local-level metacognitive judgments’ accuracy and bias, can be aggregated to depict a global-level metacognitive ability of learners that is correlated to learning outcomes. We further discuss the impact of our findings for the design of adaptive scaffolding of AIEd systems to foster metacognition.KeywordsMetacognitionSelf-regulated learningEmpirical study