Integrative processing of verbal and graphical information during re-reading predicts learning from illustrated text: an eye-movement study

Abstract

Printed or digital textbooks contain texts accompanied by various kinds of visualisation. Successful comprehension of these materials requires integrating verbal and graphical information. This study investigates the time course of processing an illustrated text through eye-tracking methodology in the school context. The aims were to identify patterns of first- and second-pass reading and to examine whether the integrative processing of text and picture during the less automatic and more purposeful second-pass reading predicts learning, after controlling for reading comprehension, prior knowledge, and self-concept. Forty-three 7th graders read an illustrated science text while their eye movements were recorded. A cluster analysis revealed two processing patterns during the first-pass reading, which differed for the time spent on the main concepts in the text and picture. During re-reading, two patterns of stronger and weaker integrative processing emerged. Integration of verbal and graphical information was revealed by the frequency of second-pass transitions from text to picture and from picture to text, and the duration of picture re-inspecting while re-reading text information (look-from text to picture) and re-reading text information while re-inspecting the visualised information (look-from picture to text). A series of hierarchical regression analyses indicated that only the patterns of integrative processing during the second-pass reading uniquely predict verbal and graphical recalls, and the transfer of knowledge. The study provides evidence that the delayed processing which integrates text and graphics contributes to text retention and the application of newly learned knowledge, over and above individual characteristics. The educational significance is outlined.