Abstract
A word’s predictability or surprisal, as determined by cloze probabilities or language models (Frank, 2013) is related to processing effort, in that less expected words take more effort to process (Hale, 2001; Lau et al., 2013). A word’s surprisal, however, may also be influenced by the non-linguistic context, such as visual cues: In the visual world paradigm (VWP), anticipatory eye movements suggest that listeners exploit the scene to predict what will be mentioned next (Altmann and Kamide, 1999). How visual context affects surprisal and processing effort, however, remains unclear. Here, we present a series of four studies providing evidence on how visually-determined probabilistic expectations for a spoken target word, as indicated by anticipatory eye movements, predict graded processing effort for that word, as assessed by a pupillometric measure (the Index of Cognitive Activity, ICA). These findings are a clear and robust demonstration that the non-linguistic context can immediately influence both lexical expectations, and surprisal-based processing effort.
Highlights
The information-theoretic concepts of entropy (Shannon, 1949) and surprisal (Hale, 2001; Levy, 2008) have gained much attention in recent psycholinguistic research since they correlate with measures of processing effort (Demberg and Keller, 2008; Smith and Levy, 2013; Frank et al, 2015) and allow for quantitative predictions in language processing
We previously suggested that our verbal constraints alone did not contain enough information to cause listeners to have lexical expectations about target nouns, resulting in the same processing effort for more or less constraining verbs and more or less plausible object nouns following those verbs
In four experiments deploying behavioral and pupillometric measures, we closely examined the effect of visual context information on target word predictability and the associated situated surprisal as well as processing effort
Summary
The information-theoretic concepts of entropy (Shannon, 1949) and surprisal (Hale, 2001; Levy, 2008) have gained much attention in recent psycholinguistic research since they correlate with measures of processing effort (Demberg and Keller, 2008; Smith and Levy, 2013; Frank et al, 2015) and allow for quantitative predictions in language processing. The surprisal values are typically used as a predictor of processing effort experienced by the listener upon encountering these words or units (Demberg and Keller, 2008; DeLong et al, 2014) This approach inherently neglects the listener and the context at a particular point in time. This is especially important when considering a real-world situation, including a visual context in which language is used and processed. The corresponding effort is examined by using a pupillary measure in the Visual World Paradigm (VWP) (Tanenhaus et al, 1995) Both the pupillary and the behavioral measures assessed effects of anticipation in the presence and absence of visual context.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
