Characterising visual context effects

Abstract

In this chapter, I will review recent research on visually situated language comprehension, and in doing so identify key characteristics of situated language comprehension. More specifically I will argue that both active visual context effects and the temporally coordinated interplay between visual attention and language comprehension are characteristic of situated comprehension, and are robust across a broad range of comprehension situations, spanning (a) different comprehension modalities (reading and spoken comprehension) and situations in which language is (versus isn’t) in accord with visual context; (b) different kinds of visual contexts (clipart depictions, photographs, and real-world objects and events); (c) speaker-based information such as eye-gaze and gestures; and (d) both concrete and abstract language. Because of their broad coverage ((a)–(d)), situated language comprehension paradigms are, in principle, well suited for developing a relatively comprehensive theory of situated language comprehension. One challenge in further specifying model predictions is the development of more detailed linking hypotheses between comprehension processes and one of the key measures used to examine situated comprehension (visual attention to objects across time).