Modeling Synchronization for Detecting Collaborative Learning Process Using a Pedagogical Conversational Agent: Investigation Using Recurrent Indicators of Gaze, Language, and Facial Expression

Abstract

AbstractConstructive interactions and knowledge integration activities are methods commonly used for learning; however, establishing successful coordination becomes a hurdle in computer-mediated collaborations. The development of systems to facilitate communication activities in such situations has been attempted, but models are still required for capturing learners’ interactions and detecting their quality. This study explored several types of verbal and nonverbal behaviors of learners that can be implemented while designing tutoring systems to effectively capture their interaction processes in scenarios where learners engage in collaborative learning mediated by a pedagogical conversational agent (PCA). This study focused on the degree of behavior recurrence of each speaker, which is considered suitable for observing levels of effectiveness. Specifically, this study focused on three indicators—gaze synchronization, language conformance, and emotional matching through facial expression—to establish a system-based index for measuring learners’ collaborative processes such as synchronization. This study experimentally examined the relationship between these indicators and the performance and process of collaborative learning among 44 learners while using PCA for facilitation. Subsequently, numerous dependent variables in the collaborative learning process were predicted using the three proposed indicators. However, no significant correlation was established between learning performance and the indicators used. These findings show that the recurrence of indicators is useful for estimating the collaborative learning process and that these indicators can be used in the development of learning support systems to trace learners’ achievements in successful interactions.