Multimodal Literacy Research Articles

Digital multimodal PechaKucha presentations in ESP: insights from students’ learning experiences

Abstract Advances in digital and audiovisual communication have contributed to the remediation and emergence of genres (Luzón and Pérez-Llantada 2022, giving rise to new discourse practices. Against this backdrop, it becomes important to review ESP pedagogy to better prepare students to communicate effectively in professional settings. For this purpose, ESP teachers should move beyond traditional literacy to embrace multimodality and develop students’ multimodal literacy. This study presents a multimodal genre-based approach to teaching PechaKucha (PK) presentations (Querol-Julián and Beltrán-Palanques 2021) and reports on students’ learning experiences. This pedagogical approach allowed students to explore the genre of PK presentations and reflect on how to construct their own version. The recipients of this approach were 127 ESP students, of which 44 voluntarily completed a survey purposely designed to explore their learning experiences regarding the preparation and performance of a digital PK presentation. Findings suggest that students made a great effort to effectively construct a digitalised PK mainly because of the format requirements. The students were faced with the challenge of making decisions about what and how to represent ideational, textual, and interpersonal meanings through a PK. Finally, this study suggests that students seemed to have become aware of the complexity of expressing meanings through the multimodal genre of PK presentations.

Left, Right then Left Again: Educators at the Intersection of Global Citizenship Education, Technology and Academic Literacies

The purpose of this article is to critically consider the roles that academic literacy facilitators fulfil in exposing students to Global Citizenship Education (GCE). In university disciplines, literacies are primary tools that students employ to interact with global events, knowledge, theories and problems. As such, multimodal literacies including written, audiovisual and cyber texts facilitate students’ access to the world through critical communication. Consequently, the authors construe GCE as disciplinary instruction that connects students to lived experiences beyond their own national borders. To demonstrate GCE, the authors employ the following methods for accessing, interpreting and generating knowledge: Firstly, a literature review is conducted. In doing so, key concepts and theories that define academic literacy and GCE are identified. Secondly, by combining reviewed literature that highlights GCE methods and scholarship pertaining to multimodal literacies, the authors make recommendations for integrating GCE into disciplines. In conclusion, the authors emphasise academic literacies, including digital discourses, as effective conduits for GCE principles and make further recommendations for future studies and methods that may be applied towards uniting literacies with international course content.

Multiliteracies and multilingualism in adult migrants

The communication paradigm has been restructured by globalisation and digitalisation, affecting the way we learn and adjusting language and literacy education. As adult migrants in urban contexts become socialised in learning environments with higher cultural complexity, a dynamic and flexible multiliterate and multilingual learning environment is needed in host countries to meet different needs. Of particular interest to the implementation of multimodal literacies is the consideration of recent advances in multilingualism proposed by the London Group mandate, which merges multilingualism and literacy. The aim of this study was to empirically investigate the interrelation of multiliteracy and multilingualism by conducting a systematic review that would shed light on how to grant ideological and implementational programmes for multilingual language education policy and practice in multiliteracy contexts. The study followed the PRISMA protocol to answer three research questions: RQ1: Has previous research incorporated multilingualism / plurilingualism into multiliteracy models for studies with adult migrants? RQ2: What research methods have been used for studies with adult migrants? RQ3: What is the quality of the research methods? Our results show that multilingualism is embedded in the various models of multiliteracy. However, the epistemological tradition in the field of multilingualism does not seem to have had a significant impact on the implementation of new educational programmes that meet the needs of adult migrants. We also found that quantitative research has rarely been used, and that there is a need to raise quality standards of research in this sub-discipline by reducing bias in the processes of designing, conducting and reporting research.

Unveiling the power of critical multimodal literacy: exploring cultural difference in children’s literature through ‘The Proudest Blue — a Story of Hijab and Family’

At a time when difference is being actively diluted in the service of standardisation and conservative views of nationhood, stories about difference, and their semiotic and multimodal elements, provide rich grounds for critical engagement. This is especially the case with stories about highly visible cultural symbols, such as the hijab, worn in many school classrooms in countries like Australia. In this paper, we explore the use of critical analysis of multimodal elements as matters with which to think about cultural difference, through the picture book, The Proudest Blue — a story of Hijab and family by Ibtihaj Muhammad with art by Hatem Aly, 2020. This story presents to young people the notion of being strong in cultural difference via the symbol of the hijab. Our analysis draws on Cappello et al.’s 2019) Critical Multimodal Literacy approach to explore four dimensions: communicating with multimodal tools; re-storying, representing and redesigning; acknowledging and shifting power relationships and leveraging multimodal resources to critique and transform socio-political realities. We identify the range of multimodal tools the author and illustrator put to use to re-story and critique common misconceptions about the wearing of the hijab as a religious symbol. Making links between multimodal composition and critical analysis is a powerful way to affirm the significance of children’s literature that addresses difference, a term often conflated with diversity which references dominant norms. It can also inspire children to story their own matters in ways that critique notions of cultural homogeneity.

Rethinking agency in literacies: Malawian children’s and teachers’ perspectives

ABSTRACT This qualitative study contributes theorised and empirically enriched insights from local practices into children’s agency. The authors trace a converging interest in multimodal literacies, postcolonial philosophies and early childhood pedagogy to document and critically engage with children’s agency in stories experienced by Malawian children in their primary schools. Agency is understood as the identities that children assign to their stories and understandings of self [Pahl, K. and Rowsell, J., 2012. Literacy and education (2nd edition), London: SAGE Publications]. Interviews with twenty-five children and two teachers from two primary schools in semi-urban Malawi and drawings from 49 children were analysed using the Sense-making Method [Weick 1995. Sensemaking in Organizations. Vol. 3. Sage]. Children’s drawings and interviews revealed children’s positive portrayals of collective agency in contexts typically associated with subdued identities (domestic chores). Gendered, age-restricted and otherwise limited agency in orchestrating diverse stories in the classroom were explicit in the teachers’ accounts. The discussion imbues contemporary early childhood studies with a new understanding of children’s agency, as a communal and context-dependent phenomenon.

Critical Multimodal Literacy and Oral Proficiency Development

This descriptive case study aimed to investigate the impact of critical multimodal literacy in English classes, which included the use of parody films and metaverse-based User-Created Content (UCC) related to disability, on Korean university EFL learners’ conventional literacy and awareness of disability. Data were collected from three participants with varying English proficiency and educational backgrounds over a two-month period, with weekly sessions lasting two and a half hours. A combination of qualitative and quantitative methods, such as questionnaires, interviews, group discussions, reflective papers, and metaverse-based UCC productions, was employed to assess the effectiveness of critical multimodal literacy in English speaking and disability education. Korean EFL students, influenced by Confucian culture, tend to hesitate when expressing their opinions publicly. In response, this research proposes using avatars to create alternative textual representations of their viewpoints instead of recording personal video presentations. This innovative approach aims to evaluate changes in students’ attitudes towards disabilities and improvements in their oral communication abilities. The overall findings indicated that (1) The critical multimodal literacy practice, employing parody films based on fairy tales in the classroom, plays a significant role in the development of conventional literacy among Korean adult EFL learners. (2) The practice of critical multimodal literacy, which involves using parody films of fairy tales that address disabilities, helps EFL learners broaden their perspectives on disabilities and modify their social behaviors. (3) Attitudes and reactions toward disabilities varied according to the individual experiences of EFL learners.

Gender-inclusive picture books in the classroom: A multimodal analysis of male subjective agencies

This article examines the representations of agentive identity of non-normative male protagonists in children's picture books in relation to the affordances of this material for classroom activities. The analysis of protagonists’ agencies in five picture books revealed varied instantiations of the New Age Boy masculinity schema. A multimodal transitivity analysis of protagonists’ agentive roles yielded a profile of individual agentive identity. Protagonists’ agentive roles when interacting with conflictive antagonist characters were compared with their agentive roles when not interacting with them. Variations were significantly associated with the social domains where the conflict took place (family, school, or community) as different ways of responding to social rejection. The findings of the study concerning the representation of protagonists’ agencies provide young readers with models of action in situations of social rejection that can help make more informed decisions for guiding the selection of gender-diverse reading material and of classroom activities of critical thinking and multimodal literacy.

Let's Learn from Them: Using the Integrative Multimodal Literacy Assessment Tool to Support Instruction for Young Children

AbstractMany children bring rich multimodal literacy practices into their classrooms, with experience weaving between paper and digital texts. In order to support children's multimodal meaning making, teachers need first to understand the multimodal practices that children bring into the classroom in order to then develop instruction that builds on, and extends, children's existing literacy practices. This article presents a tool to assess young children's multimodal literacies. The Integrative Multimodal Literacy Assessment (IMLA) evaluates children's multimodal meaning making through a questionnaire and performance activity. In this article, we first describe the tool and explain how to administer it in diverse classroom contexts. We then share findings from a pilot study, using the IMLA, conducted in two urban preschools as an example of how teachers might use the tool in their own classrooms.

Digitally transformed lifelong multilingual language learning: affective processes underlying the development of multilingual competence

This paper reflects on the issues of informal and non-institutional practices of language learning in digital environments. In this qualitative and longitudinal case study, authentic language materials were used by two adult learners for their development of Italian language skills and multilingual competencies through Personal Multilingual Dictionary Design (Italian/English/Greek/Serbian). The authentic materials included three editions (500 episodes) of the most famous show “É sempre mezzogiorno” (“It is Always Noon”) by the Italian public television service “Rai” (2020/2023). The choice of the video materials, as learning materials, was based on the interests and personal preferences of the learners, highly interested, involved and committed to language learning and the development of multilingual literacy. Affective processes underlying the development of multilingual competence through collaborative learning tasks are examined and discussed. In conclusion, several areas of multimodal literacies are highlighted, with potential guidance in envisioning future research processes. / Keywords: authentic video content, informal foreign language learning, Italian language, multilingual competence

Family literacies as peacemaking: Representations in Children’s literature

This paper presents an analysis of the discursive contours of peacemaking within families as represented in children’s literature. We turned to the Jane Addams Children’s Book Award (JACBA), which recognizes literature that engages children in thinking about peace and social justice. We analyzed the 2015-2021 collection of twenty-six award winning books for younger children. We found at least ten pathways of peacemaking that families engage in through their multimodal literacy practices. Drawing on family-centric peacemaking episodes, we present analytically rich portraits of how peacemaking unfolds at textual, ideational, and interpersonal levels across time, space, and generations. We argue this analysis provides us insight into the historical, political, and social realities that inform students and families’ lives in early childhood literacy classrooms.

Using Digital Game-based Learning to Foster Multimodal Literacy in Left-behind Children of Migrant Workers: Possibilities and Challenges

Digital games offer opportunities for the youth’s development of multimodal literacy required in the technology-driven world. However, both digital games and multimodal literacy are undervalued in rural contexts characterized by migrant parents and left-behind children. This qualitative case study conceptualized systematic guidance to foster students’ multimodal literacy through digital games and trialled its possibilities and challenges in enhancing left-behind children of migrant workers in a suburban school. Sixteen student participants engaged in critical game analysis and game video design in a 16-session classroom-based program, with the guidance of the researchers and two local teachers. Multiple data-collection methods were applied to capture students’ learning progress aligned with the teachers’ perceptions. Thematic analysis and multimodal analysis were utilized to process data. The findings indicate that this instructional approach deepened and broadened the students’ critical viewing and application of multimodal texts and enhanced their proficiency in leveraging digital tools for multimodal communication. The challenges include students’ varied competencies and motivation, time constraints, and limited access to digital devices. We advocate for integrating DGBL in literacy classrooms, thus catering to left-home students’ demands and social needs in the digital age. Meanwhile, joint efforts are necessitated to address perennial challenges in this education reform and inequities within the process.

Book Review of Disciplinary Literacy and Gamified Learning in Elementary Classrooms: Questing Through Time and Space

Disciplinary Literacy and Gamified Learning in Elementary Classrooms: Questing Through Time and Space is a handbook for launching quest-based learning activities in the elementary classroom. The work collates resources that support content area learning and multi-modal literacy. Students are centered in the learning activities as creators and players, thus making learning an active and engaging experience. Each unit articulates the learning standards, lesson objectives, and evaluation methods, effectively serving as a paradigm of best practices in teaching. This resource is complemented by an online resource hub for each unit of study, which includes a supplementary website, videos, and printable lesson plans.

Reading landscapes and writing nature

This reflective article discusses lessons learned when Reading Landscapes & Writing Nature, an annual collaboration between a National Writing Project site and Weir Farm National Historical Park, migrated online in 2020 as a result of the Covid-19 pandemic. The organizations have used critical pedagogies of place (Gruenewald, 2003) since 2017 to guide teacher writing workshops, and reimagined the professional development in digital spaces with multimodal literacies (Kinloch, 2009; Kress, 2003). This including 360 photospheres and Padlet as tools to expand educators’ understandings of literacy, wellness, and place.

Condensing Meaning

In this chapter we address the challenge of interpreting and teaching complex infographics of the kind read and viewed in secondary school science in Australia. Inspired by the work of Bateman and his colleagues we adopt a complementary bottom-up and top-down perspective – analysing infographics bottom-up in terms of general gestalt grouping principles (for both micro- and macro-groups) and interpreting images top-down with respect to abductions based on modelling of the relevant field (as activity, item, and property), in Systemic Functional Linguistic theory (SFL). We then compare two infographics in detail, considering how mitosis is intermodally construed as activity and composition in verbiage and image. The key finding arising from this analysis is that in spite of the complexity of the infographics, considerable information has to be abduced, drawing on additional knowledge of the field – information that the co-text in the textbooks accompanying these infographics may not make explicit. This highlights the importance of multimodal literacy pedagogy as a crucial part of science education. We show how such a pedagogy can address the complexity of reading infographics in conjunction with verbal co-text. In this kind of carefully scaffolded methodology, the synoptic view of complex knowledge aggregation provided by infographics can play a significant role in the dynamics of knowledge building.

Students’ Perceptions of Using Roblox in Multimodal Literacy Practices in Teaching and Learning English

The need for digital technology involvement in education is increasingly apparent, so learning techniques are required to be prepared in various digital formats. Using digital learning materials enables teachers to implement the current concept of literacy, i.e., multimodal literacy. This study aims to find out the students' need for digital literacy and their perceptions of using the Roblox game as the learning media to improve their multimodal literacy. This descriptive study used a survey method, taking the second-year junior high school students in the 2021-2022 academic year in one of the state junior high schools in Medan, Indonesia as the respondents. The data were collected using two sets of closed-ended questionnaires. The data obtained from the first questionnaire were analyzed using a forced choice technique for yes or no answers, while the data obtained from the second questionnaire were analyzed using a 4-Point Likert Scale. The results indicate the students' need for digital literacy and their good perceptions of using Roblox as the learning media in improving their English multimodal literacy. It is concluded that teaching English at junior high school is expected to be designed in a digital format by involving games as the media, facilitating the students to improve their English multimodal literacy.

Teaching Multimodal Literacies with Digital Technologies and Augmented Reality: A Cluster Analysis of Australian Teachers’ TPACK

Despite the proliferation of augmented reality (AR) apps, Australian primary teachers have yet to use them widely for the teaching of multimodal literacies. Conceptualising teachers’ knowledge of using digital technologies to teach multimodal literacies as a form of technological pedagogical content knowledge or TPACK(ML), this study examined teacher differences through a cluster analysis of survey responses collected from a sample of 142 Australian primary school teachers. Two distinct clusters of teachers were derived. The first cluster with lower TPACK(ML) comprised teachers with lower self-reported confidence in facilitating new cultures of learning that are participatory and technology-driven in nature. In their open-ended survey responses, these teachers shared their unfamiliarity with AR, as well as concerns about their personal technical competency and how AR could be integrated into the curriculum. The second cluster of teachers rated themselves higher in TPACK(ML) and in how they used technology to support language learning pedagogies. They were able to propose different pedagogical strategies to engage students’ multimodal literacies meaningfully with AR in their open-ended survey responses. The implications of the study’s findings were discussed, and recommendations were proposed for designing and sustaining differentiated forms of teacher professional development for teaching multimodal literacies with emergent digital technologies.

Critical Literacy In An EFL Learning Virtual Course

The objective of the present investigation is related to describing how a group of intermediate-level students responds to a pedagogical implementation of critical literacy in an online EFL course at a language institute. The study has a qualitative approach and descriptive level. The sample consisted of six female students, all of them 15 and 16-year-old high school students; with a command of the English language at an intermediate level B1. This study considers four data collection instruments: classroom observations - video recordings and teachers' diaries, semi-structured open interviews with the aim of capturing the representations, thoughts, and conceptions that students have built on social problems through the approach of critical literacy implemented to teach English in a virtual learning environment. Among the main findings is that from the development of multimodal literacy, students achieved more structured knowledge in arguments; also more autonomous learning as well as a clear increase in interaction and consensus among students; finally, the mastery of the sources and the collection of information for learning increased. Overall, participants responded effectively to the development of critical literacy skills, through a gradual and satisfactory learning process.

Technology-Assisted Reading Instruction for English Language Learners: A Methodological Review

Purpose By methodologically reviewing the literature on Technology-Assisted Reading Instruction (TARI) for English Language Learners (ELLs) in K-12 settings, this study aims to advance the knowledge of TARI research and guide future research directions. Design/Approach/Methods This study examines 32 peer-reviewed journal articles published between 2000 and 2020. Each article was analyzed for research objectives, reading skills, learner demographics and contexts, theoretical frameworks, research designs, and outcomes. Findings A review of the literature reveals a strong focus on evaluating the efficacy of TARI through summative assessments, but limited attention to learners’ literacy practices and interactive engagement in digital contexts. Additionally, there is a marked lack of alternative approaches to assessing the effectiveness of technology in promoting ELLs’ reading progression and a continued emphasis on developing traditional literacy over multimodal literacy. TARI can enhance reading motivation, foster collaborative learning, provide scaffolding, improve reading performance, and expand semiotic resources. Originality/Value As one of the first comprehensive methodological reviews of TARI, this study elucidates ELL reading education in the era of multiliteracy. Results have implications for technology-mediated education, which expanded rapidly during the COVID-19 pandemic.

Celebrate with me: a black adolescent girl’s speculative multimodal design of intersectional college and career futures

PurposeThis paper aims to demonstrate how Alayah, a 16-year-old African American girl, leverages multiple expressive modes for intersectional self-representation as speculative design. Here, speculative design refers to a multimodal composition (i.e. digital collage) which leverages multiple expressive modes for intersectional self-celebration in possible futures.Design/methodology/approachInformed by intersectional multimodal literacy frameworks and analyses, this paper addresses the question, “How does Alayah represent her college and career futures in her speculative multimodal design? To address this question, the author analyzed Alayah's digital collage using an intersectional multimodal analysis template complemented by a thematic analysis of her interview data and the narrated explanation of her collage.FindingsIn a speculative design composed of 15 images and words, Alayah agentively determined and critically celebrated her intersectional college and career futures through four interrelated themes: Black girl affirmation; Collegiate success; “Sweet” work; and Black livingness.Originality/valueBy centering Black girls’ speculative multimodal designs in college and career curricula, ELA educators (re)imagine college and career pedagogies to critically celebrate Black adolescent girls as intelligent, empowered and literate young women worthy of the futures that they desire.

Multimodal Literacy in a New Era of Educational Technology: Comparing Points of View in Animations of Children's and Adult Literature

Purpose The paper shows the interpretive impact of different constructions of the point of view available to the reader/viewer in book and animated movie versions of a children's picture book, a novel for pre-adolescents/early teenagers, and a graphic novel for adolescents and adults. Design/Approach/Methods Excerpts from book and animated movie versions of the same story are compared using multimodal analysis of interpersonal meaning to show how the reader/viewer is positioned in relation to the characters in each version, complemented by analyses of ideational meaning to show the effect of point of view on interpretive possibilities. Findings Focusing mainly on multimodal construction of point of view, the analyses show how interpretive possibilities of ostensibly the same story are significantly reconfigured in animated adaptations compared with book versions even when the verbal narrative remains substantially unchanged. Originality/Value The study shows that it is crucial to students’ critical appreciation of, and their creative contribution to, their evolving digital literary culture that in this new era of educational technology, attention in literacy and literary education focuses on developing understandings of digital multimodal narrative art, and that animated movie adaptations are not presented pedagogically as isomorphic with, or simply adjunct to, corresponding book versions.