Intermodality and visual literacy: exploring visual-verbal instantiation in children’s picture books on coronavirus

Abstract

The study examines the intermodality in children’s picture books, with a special focus on the visual-verbal meaning instantiation in young learners’ visual literacy readings. Picture books serve as one of the main platforms and channels that prepare children for early literacy education. Coronavirus as a key theme children have encountered during the COVID-19 pandemic period has been widely discussed in picture books for different educational purposes. This study employs systemic functional linguistics (SFL)-based multimodal discourse analysis and draws upon Painter, Martin, and Unsworth’s (2013) visual narrative analytic framework to explore the intermodal relationships between the visual and verbal meaning systems and how each semiotic system is represented and instantiated in the picture books as bimodal texts. Coronavirus: A book for children (2020) and My hero is you: How kids can fight COVID-19 (2020), the two most widely disseminated picture books on Coronavirus, were chosen for case study and comparative analysis. Through these, the intermodal meaning making process that features in children’s literature is examined and synthesised from the three visual analytical perspectives of the representations, relationships, and organisations in the social semiotic construction to facilitate children’s ability to read and comprehend the COVID-related visual information in integration with the verbal semiotics. The findings reveal the visual images function as visual recontextualization for the verbal realisations and provide imagistic contextualisation for the textualized storytelling.