Using eye-tracking to assess sourcing during multiple document reading: A critical analysis

Abstract

During the last 15 years, there have been some efforts to extend the use of eye-tracking to researching reading in complex contexts, such as the reading of multiple documents. The research community involved in this extension has been interested in higher-order comprehension processes occurring in complex reading contexts, such as sourcing, defined as the processes of attending to, representing, evaluating, and using available or accessible information about the sources of textual content. In this article, we argue that extending eye-tracking research to investigate more complex reading contexts has been made without critically reflecting on its reliability and validity in those contexts. Specifically, because eye-tracking captures automatic as well as conscious processes, it is currently an open question how reliably and consistently eye-tracking captures the strategic sourcing processes that take place during multiple document reading, in particular compared to subjective methods that mainly target conscious processes, such as interviews. We compared sourcing indicators based on eye-tracking measures to sourcing indicated by a post-reading interview. Results suggested that current eye-tracking indices of sourcing are not universally valid and reliable measures, and that simpler methods, such as interviews, may be more suited to assess strategic sourcing during multiple document reading.