Abstract
Predictability is known to modulate semantic processing in language, but it is unclear to what extent this applies for other modalities. Here we ask whether similar cognitive processes are at play in predicting upcoming events in a non-verbal visual narrative. Typically developing adults viewed comics sequences in which a target panel was highly predictable (“high cloze”), less predictable (“low cloze”), or incongruent with the preceding narrative context (“anomalous”) during EEG recording. High and low predictable sequences were determined by a pretest where participants assessed “what happened next?”, resulting in cloze probability scores for sequence outcomes comparable to those used to measure predictability in sentence processing. Through both factorial and correlational analyses, we show a significant modulation of neural responses by cloze such that N400 effects are diminished as a target panel in a comic sequence becomes more predictable. Predictability thus appears to play a similar role in non-verbal comprehension of sequential images as in language comprehension, providing further evidence for the domain generality of semantic processing in the brain.
Highlights
Predictability is known to modulate semantic processing in language, but it is unclear to what extent this applies for other modalities
High-cloze words generate a reduced N400 compared to low-cloze words, suggesting that the greater the predictability of a word given the constraints of the preceding sentence context, the smaller the N400 amplitude
By quantifying and manipulating cloze in this way, we aim to demonstrate that predictability in visual narrative sequences modulates the N400 response in similar ways as in language, thereby providing further evidence that visual and linguistic narrative processing rely on similar cognitive mechanisms
Summary
Predictability is known to modulate semantic processing in language, but it is unclear to what extent this applies for other modalities. High and low predictable sequences were determined by a pretest where participants assessed “what happened next?”, resulting in cloze probability scores for sequence outcomes comparable to those used to measure predictability in sentence processing Through both factorial and correlational analyses, we show a significant modulation of neural responses by cloze such that N400 effects are diminished as a target panel in a comic sequence becomes more predictable. As in sentences, incongruous or unexpected panels in a visual narrative sequence have been shown to evoke larger N400s than congruous panels[22,23,24] Given these similarities between visual and linguistic semantic processing, and the wealth of research examining predictability in the linguistic domain, it is curious that only a few studies to date have discussed the role of prediction in comprehending visual sequences[23,25,26,27,28]. This work clearly highlights the importance of prediction in visual event comprehension, none of these studies by Zacks and colleagues manipulate predictability by, for example, including predictable or unpredictable events or classifying stimuli based on cloze ratings
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
