Adaptive multimedia: Using gaze-contingent instructional guidance to provide personalized processing support

Abstract

The goal of the study was to develop an adaptive, gaze-contingent learning environment that would support learners in their information-processing behavior when learning from illustrated texts. To this end, three experiments were conducted. In Experiment 1 (N = 32) three groups of learners were identified based on differences in their eye movements obtained while they were learning with a non-adaptive multimedia learning environment. The group of learners who displayed longer fixations times and higher fixations counts on text and pictures as well as more text-picture transitions had better learning outcomes than a group with a less intense information-processing behavior. These findings were used to develop a gaze-contingent adaptive system. It analyzes learners’ eye movements during learning in real time and - in case of poor information processing (i.e., behavior similar to that of the unsuccessful learner group in Experiment 1) - alters the presentation of the materials in a way that is expected to trigger a more adequate processing (e.g., by highlighting relations between text and pictures). In Experiment 2 (N = 79) and Experiment 3 (N = 62) the adaptive multimedia learning system was compared to a non-adaptive, static presentation of the same materials. Experiment 2 showed no differences between both learning systems in terms of learning outcome. In Experiment 3, where the thresholds for adaptive responses were slightly modified, the gaze-based adaptive system hindered learners with weaker cognitive prerequisites, but tended to support learners with stronger cognitive prerequisites. Possible reasons are discussed and future research directions suggested.