The effects of social and cognitive cues on learning comprehension, eye-gaze pattern, and cognitive load in video instruction

Abstract

Students experience challenges when understanding visual information in multimedia learning. Specifically, immersive multimedia environments, such as virtual reality increase the likelihood that students undergo distractions in which information seeking during system-paced instruction occurred. Although previous studies have reviewed various cue designs to yield students’ higher attention, skepticism still exists regarding which ways cue designs can support their learning comprehension in video instruction. For this study, we sampled a total of 64 undergraduates in a university. Using video instruction performed by an animated pedagogical agent (APA), this study examined the effect of social (i.e., an APA’s conversational gestures) and cognitive (i.e., visual cue) cues on students’ learning comprehension and eye-gaze data within types of visual information (text and pictorial). Also, this study investigated how both cues promoted students’ cognitive load overall. Specific to text information processing, the results of the study confirmed that the negative prime effect of social cues undermined students’ learning comprehension and increased their cognitive load, whereas cognitive cues appeared to be supportive in video instruction. Also, this study found that students’ different visual-attention patterns appeared in pictorial information processing. In terms of pictorial information processing, the study finding implies that whereas social cues caused visual distractions and lowered learning comprehension, cognitive cues as visual cues helped learners to integrate pictorial information via visuospatial clues. Conclusively, we reported several design implications derived from the study findings.