Abstract
Visual narratives sometimes depict successive images with different characters in the same physical space; corpus analysis has revealed that this occurs more often in Japanese manga than American comics. We used event-related brain potentials to determine whether comprehension of “visual narrative conjunctions” invokes not only incremental mental updating as traditionally assumed, but also, as we propose, “grammatical” combinatoric processing. We thus crossed (non)/conjunction sequences with character (in)/congruity. Conjunctions elicited a larger anterior negativity (300–500 ms) than nonconjunctions, regardless of congruity, implicating “grammatical” processes. Conjunction and incongruity both elicited larger P600s (500–700 ms), indexing updating. Both conjunction effects were modulated by participants’ frequency of reading manga while growing up. Greater anterior negativity in frequent manga readers suggests more reliance on combinatoric processing; larger P600 effects in infrequent manga readers suggest more resources devoted to mental updating. As in language comprehension, it seems that processing conjunctions in visual narratives is not just mental updating but also partly grammatical, conditioned by comic readers’ experience with specific visual narrative structures.
Highlights
Drawn sequential images are ubiquitous in human communication; they extend throughout human history and across cultures from cave paintings and scrolls to contemporary comics and storyboards that guide storytelling in films (McCloud, 1993)
Consistent with the VNG framework, but not the canonical semantic-updating account, we hypothesized that these two visual narrative comprehension components would be modulated by participants’ experience with particular visual narrative constructions, which based on corpus analysis is more prevalent in Japanese manga than American comics (Cohn, 2011, 2013a, in press; Cohn, Taylor-Weiner, et al, 2012b)
Altogether, our findings indicate that visual narrative comprehension involves multiple interacting processes: here, updating of a mental model and a combinatorial narrative grammar
Summary
Drawn sequential images are ubiquitous in human communication; they extend throughout human history and across cultures from cave paintings and scrolls to contemporary comics and storyboards that guide storytelling in films (McCloud, 1993). Image sequencing tasks are staples within IQ assessment (Kaufman & Lichtenberger, 2006; Ramos & Die, 1986), and a growing movement has advocated using visual narratives such as comics in education (Short, Randolph-Seng, & McKenny, 2013) This prevalence of sequential images is underlined by a belief that. Cohn and Kutas Cognitive Research: Principles and Implications (2017) 2:27 their comprehension is universal and fairly transparent (Berliner & Cohen, 2011; Levin & Simons, 2000; McCloud, 1993) Given these diverse real-world contexts, we ask: how uniform is visual narrative processing?. This presumes that sequential image comprehension engages basic cognitive processing (perceptual and semantic systems) which operates across individuals
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call