Multimodal Meaning-making Research Articles

Teacher Education during the COVID-19 Lockdown: Insights from a Formative Intervention Approach Involving Online Feedback

This paper examines preservice teachers’ perspectives on assessment feedback developed in a teacher education course during the first lockdown due to the COVID-19 pandemic. As initially negotiated with students, feedback was learner-centred and involved a formative intervention approach applied iteratively by the teacher educator over the course of one semester. Although such feedback was initially face-to-face, it had to be given exclusively online following the unexpected closure of the university. Analysis of student teachers’ perspectives, which were collected through an online questionnaire completed after their final assessment, reveals both positive and critical aspects regarding the feedback provided by the teacher educator. While reaffirming the significance of feedback as a crucial element for learning in online teacher education contexts, the findings also show that the clarity, affective bonding and multimodal meaning-making involved in face-to-face interaction are particularly challenging when the communication of feedback is digitally mediated. The implications and limitations of such findings are discussed.

Everyday stories in a forest: Multimodal meaning-making with Ojibwe Elders, young people, language, and place

Everyday stories in a forest: Multimodal meaning-making with Ojibwe Elders, young people, language, and place

The EFL-YouTube remix: Empowering multimodal and computational literacies for EFL purposes

Because innovations in media technologies affect all aspects of our daily lives, multimodal and computational literacies are becoming increasingly important as a way to empower English as a Foreign Language (EFL) learners in their second language practices. Content-sharing platforms, such as YouTube, provide instant access to authentic language environments in a wide variety of contexts. While many learners operate emergent software services intuitively and with great ease, their potential benefits for language learning remain widely unexplored. This paper examines multimodal and computational meaning-making practices in EFL contexts from a theoretical and empirical perspective. More specifically, it focuses on how competencies in multimodal meaning-making and software programming can be combined with the lexical approach to language learning when repurposing YouTube videos to teach English idioms and phrases. Based on a prototype language learning website designed and implemented by students at the University of Klagenfurt and improved by students from Bundeswehr University Munich, the paper identifies fundamental literacy skills required for a critical and self-determined application of software services in English language classrooms in terms of four emergent digital meaning-making practices: multilinear choices, multimodal meaning-making, data curation and procedural reasoning. The prototype language learning website that evolved as part of this project can be accessed at www.lingo.farm.

Towards Emancipatory L2 Instruction: Exploring Significant Learning Outcomes from Collaborative Digital Storytelling

<p style="text-align:justify">Digital storytelling has undergone extensive study in different content-areas, but its naturally-combined use with collaborative writing for skills development, and reflective practice remains underresearched in pre-service EFL teacher education. This study undertook joint tech-enhanced retelling of L2 texts by 56 Turkish EFL teacher candidates, rubric-based peer and teacher assessment of final products, comparative analysis of complexity, accuracy, and fluency (CAF) between outliers, and process evaluation using the significant learning taxonomy to explore impacts on L2 writing performance, academic learning, and personal growth. Despite assigning lower scores than the teacher-assessors, especially to the top-performers, the majority of peers successfully fulfilled the job, effectively performed the future reviewer role, and positively reacted to co-construction, and technology integration. CAF and reflection analyses indicated that the biggest difference between the highest- and lowest-scoring groups lay in grammatical accuracy, and lack of mutual interaction could account for the less cooperative group’s poorer performance. The classification of their post-task responses into six kinds of learning gains (foundational knowledge, application, integration, human dimension, caring, and learning how to learn) also revealed that their collaborative digital storytelling experience (CDS) elicited more procedural, critical, creative and practical thinking on the academic learning front, while disciplinary and integrative thinking may have declined due to more immediate preoccupation with task achievement. Their critical thinking was mainly organised around consensus-reaching, fluctuating membership, and logistical challenges, and most demonstrated a clear understanding of the role of positive group dynamics in group outcomes. Despite heightened awareness of the performance-boosting, character-forming, and motivational benefits of collective scaffolding and multimodal meaning-making, a minority could also discern the instrumentality of innovative teaching techniques in their future classroom practices.</p>

Multiliteracies in EL Curriculum Implementation and Pedagogy in Multilingual Classrooms of Southeast Asia

In keeping up with the current push for multiliteracies in ELT, the secondary school EL curriculum of Singapore has placed an emphasis on multiliteracies. Students are encouraged to engage with ‘rich language’ through a range of semiotic resources, including the use of multimodal texts. Drawing on the framework of multiliteracies pedagogy that integrates the four components of situated practice, overt instruction, critical framing and transformed practice (New London Group, 1996), this article examines multimodal meaning-making in the enacted EL curriculum in Singapore’s multilingual classrooms. By looking at how visual literacy is taught and how multimodal texts are used in curriculum implementation and pedagogy, I hope to demonstrate that the framework of multiliteracies pedagogy adopted or adapted, has provided a rich environment for students to create engaging and interactive learning opportunities for themselves. I also hope to showcase how visualization training to develop the mind’s eye through carefully designed language learning tasks can enhance students’ visual literacy in an increasingly multi-modal, multi-dimensional world where they are surrounded by an array of semiotic resources across language and culture.

The Multimodal Meaning-Making Process in Educational Design Team Meetings

The aim of this study is to contribute to a better understanding of the nuances of multimodal communication through which educational designers construct shared meanings and ideas. Educational design is a broad and multi-faceted area. Meaning-making in face-to-face educational design meetings is a complex process but is yet to gain attention as a topic of research. Most educational design projects involve various stakeholders, including professional designers and people who offer distinct expertise according to the needs of the project. Effective communication is imperative in such environments. We need a thorough understanding of how educational designers work together before we theorise their practice and attempt to create prescriptive tools and process models. The analysis of the meaning-making process reported in this paper shows that educational design is accomplished through intricate verbal and non-verbal interaction with various tools, resources, and representations. In particular, the results show how educational designers reconstructed pre-created visual design artefacts through drawing inscriptions over them – blending digital and non-digital artefacts. In addition, this study shows that gestures are indispensable to the meaning-making process in educational design team meetings. It is important to build a body of knowledge in this area that supports the practice and research in educational design. This paper would be of primary interest to researchers who study the way in which educational design is accomplished. Beyond this, those who work as educational designers or those who work with educational designers, might benefit from an awareness of the different interactional tools at their disposal.

Answering the world: young children’s running and rolling as more-than-human multimodal meaning making

This paper makes a case for a view of young children’s meaning-making in which human actants are not separate from, but deeply entwined in, a more-than-human world. In order to interrogate the more-than-human processes through which multimodal meaning-making emerges, we focus on meaning-making through running and rolling that we have observed in early childhood settings in Finland and the UK. In doing so, we rethink the process of bringing-into-relation that underpin multimodal literacy practices. Ingold’s notion of correspondence is offered as a generative way to conceptualize the interplay between human and nonhuman elements as they ‘make themselves intelligible to each other’ (p.97). We show how posthuman theory offers the possibility for reconceptualising emergence and intentionality, within young children’s meaning-making.

‘Please could you stop the noise’: The grammar of multimodal meaning-making in Radiohead’s ‘Paranoid Android’

This article uses Zbikowski’s theory of ‘musical grammar’ to analyse Radiohead’s song ‘Paranoid Android’ from their 1997 album OK Computer. Invoking the close structural and compositional parallels between language and music, Zbikowski’s approach appropriates some of the core elements of cognitive linguistics to provide a means of ‘translating’ music into meaning-bearing conceptual structures via the construction of ‘sonic analogues’, which are a type of conceptual construct formed when incoming perceptual information is compared to existing cognitive knowledge stored as image schemas. The result is an analysis of the interactions between the linguistic and aural constructions of a multimodal text that not only sheds new light on the text’s meaning-making devices, but also endeavours to unlock the strategies through which such distinctive semiotic modes act and interact within texts to create meaning potential.

Multimodality in Language Education – Implications for Teaching

The aim of this article is to discuss how a multimodal approach to meaning-making can contribute to language education and how multimodal meaning-making is supported in the Swedish curricula. Considering the rapid digitalization of contemporary communication, the aim and content of language education has been challenged. The article describes contemporary communication and meaning-making from a socio-semiotic, multimodal approach. Based on an example from a poetry assignment and students’ solutions in a Swedish as a first language framework, we want to discuss the possibilities and challenges for meaning-making and teaching, while opening up the subject of Swedish for multimodality. Two poems are viewed from a multimodal perspective showing the usage of different modes and media. Based on this the article investigates the support in the curricula for multimodal meaning-making. The article concludes by stressing the importance of recognizing multimodal meaning-making as learning in language education.

Visual formation of science content in young students’ multimodal compositions – seven content representations

This article explores visual formation of science content in multimodal compositions (text and image) made by students through their second school year, and aims to contribute to the body of metalanguage for explicit description and interpretation of this kind of meaning-making. The social semiotic analysis examined both visual and multimodal meaning-making, and resulted in descriptions of seven content representations. These are representing content as theory (content reduced to its most general aspects), as a natural experience (as would be seen with one’s own eyes), as an event (movement, process, perspectives), as art (abstract and/or sensory, vibrant colours and/or experimental shapes), as a person (anthropomorphization of content), as something attitude-evoking (explicitly revealing the author’s attitude towards the content) and as cultural heritage (metaphoric meaning instead of scientific). Among analysed student compositions, these content representations occur both separately and in combinations with each other. Each content representation and its specific combinations of semiotic resources are presented in detail and explained with illustrative examples. Finally, the representations are discussed in terms of their relevance for teaching and assessing visually oriented science literacy in the early school years, and for pre- and in-service teacher training.

Multimodal meaning-making during play in two Northern Canadian Indigenous kindergarten classrooms

This research explores young Indigenous children’s multimodal meaning-making to carry out social intentions in dramatic and construction/materials play settings. The participants are two teachers and 21 children from two Northern Canadian Indigenous communities. Underpinned by social semiotic theory, the research involves inductive analyses of six videos of children’s play. Our findings show a richness in Indigenous children’s meaning-making, as they used verbal and non-verbal modes to carry out 26 specific social intentions that we grouped into four broad social intention categories: Getting Along, Expressing Emotion/Interest, Satisfying Own Needs and Directing. The social intention carried out most frequently was showing interest in an activity. Participating children were more likely to use non-verbal modes, particularly in construction/materials play contexts. They also combined verbal and non-verbal modes to achieve their social intentions, but did not use verbal modes exclusively. When children used verbal modes to any great extent, it was primarily in a dramatic play context where the teacher took a role in children’s dramatic play. Our research indicates a need for greater attention by educators, curriculum developers and researchers to multimodal meaning-making in Indigenous children’s play, given the cultural importance of non-verbal communication and participating Indigenous children’s remarkable multimodal meaning-making during play.

Interlocutor–child Interactions: Supporting Children's Creativity in Graphic-narrative-embodied Play

FOSTERING CREATIVITY IN CHILDREN'S learning is prioritised in a number of early childhood education framework documents across the world. Despite this emphasis, the educator's role in supporting children's creativity is often mitigated due to lack of understanding about the nature of creativity and how to appropriately provide support. This paper presents a practitioner-based case study of children's graphic-narrative-embodied play experiences through interlocutor–child interactions in one early childhood setting in Melbourne, Australia. The study aimed to investigate how one-to-one creative dialogues support children's drawing, talking and gesturing. Three children's graphic-narrative-embodied play and interlocutor–child interactions were video-recorded, transcribed and analysed using an interpretivist paradigm. The analysis process was guided by sociocultural theories and pre-existing frameworks on children's creative dispositions, thinking styles and creative processes in multimodal meaning-making. Key findings include conditions that favour creativity in children's graphic-narrative-embodied play and approaches to co-creating this with children.

New(s) Readers: Multimodal Meaning-Making in AJ+ Captioned Video

New(s) Readers: Multimodal Meaning-Making in AJ+ Captioned Video

Socio-semiotic patterns in digital meaning-making: semiotic choice as indicator of communicative experience

ABSTRACTAccess to digital technology in the classroom enables the composition and organization of ideas on screen with a variety of semiotic systems of different modes and media. This study explores patterns of communication and preference of design in digital meaning-making of twelve 7–8 years old students. Meanings were shaped in complex uses and combinations of modes and media engaging the students in negotiation of meanings where both affordances of technology, semiotic resources in the class as well as the student's prior language and cultural experiences had impact on their choices and designing of texts on screen. The opportunity to make own choices of designs revealed their designing strategies with a predominant focus on writing as the mode of dissemination and examples of semiotic work with preference to visual resources, demonstrating students’ communicative experiences. Categorizing the selection of modes and applying semiotic grammar made the means the students used to communicate meaning evident and visible, providing the meta-tools needed to understand the semiotic work of students in a broader and informed accounts for the multimodal and digital meaning-making they demonstrate – a valuable insight in regard to literacy pedagogy.

Meaning between, in and around words, gestures and postures – multimodal meaning-making in children's classroom discourse

The view of language from a social semiotic perspective is clear. Language is one of many semiotic resources we employ in our communicative practices. That is to say that while language is at times dominant, it always operates within a multimodal frame and furthermore, at times modes other than language are dominant. The 2014 National Curriculum for the UK, on the other hand, values pupils’ face-to-face classroom interaction in terms of standard spoken English (i.e. in terms of the mode of language alone). This paper offers examples demonstrating how embodied modes such as gesture, posture, facial expression, gaze and haptics work in conjunction with speech in children's collaborative construction of knowledge. In other words, what may have been previously conceived as gaps and silences – often interpreted as an absence of language – are in fact instantiations of the work of semiotic modes other than language. In order to consider this closely, this paper offers evidence from a multimodal micro-analysis of pupil-to-pupil, face-to-face interaction in one science lesson in a Year Five UK Primary classroom. It demonstrates how children's meaning-making is achieved through apt use of all available semiotic resources.

Staging Encounters: Assessing the Performance of Context in Students’ Multimodal Writing

Abstract Acknowledging the nearly universal recognition of the significance of context to writing and writing assessment as well as the bedeviling nature of this vague term, this article sketches a theory of context that is at once appropriate to multimodal meaning-making and practicable for multimodal writing assessment. This theory, which insists that texts and contexts are mutually constitutive, reveals the ways in which multimodal texts perform contexts of production and reception. Using a student example, the article models how to assess the performance of context in students’ multimodal writing.

아동의 그림일기에 나타난 글과 그림 간의 의미 구성 방식

Digital technology has advanced rapidly and it is anticipated that multimodal ways of meaning-making will become increasingly important. Consequently, teaching multimodal literacies is becoming a major issue in education. This study focuses on the use of picture diaries as a means of teaching multimodal literacies. Picture diaries are one of the basic and unique multimodal texts used in lower elementary level classes in Korea. A further advantage is that it is a promising text model which can be taught in unplugged ways. In order to explore the educational implications of using such picture diaries, this study sought to analyze the ways in which twenty four graders in an elementary school constructed meaning with written language and pictures in composing picture diaries. 251 picture diaries composed during several months of their grade period were analyzed based on the constant comparative method. The results indicated that the students utilized both written language and pictures in diverse and creative ways to provide their audience with more comprehensive meaning. These results indicate that teachers need to consider their students as active multimodal meaning-makers and provide their students with more opportunities to practice multimodal meaning-making and share their experiences.

Instantiation of multimodal semiotic systems in science classroom discourse

Science classroom discourse is inherently multimodal in that scientific meanings are made through an integration of multiple semiotic systems (e.g., language, diagrams, equations). Although some studies have described this multimodal nature, few have examined and explained the relationship between the integration of multiple semiotic systems and the instantiation of science content knowledge. Based on the notion of instantiation from systemic functional theory, this paper proposes a theoretical framework to account for how specific scientific meanings are instantiated in local acts of multimodal meaning-making. Using data from a series of nanoscience lessons, four exemplars of middle school students’ acts of meaning-making were analyzed to illustrate the theoretical framework. Specifically, each exemplar illustrates how a distinctive ideational meaning was instantiated through the co-deployment of verbal, visual, and gestural modes used in the students’ explanations of a physical phenomenon. This will be useful in furthering our understanding of the structure of science classroom discourse and its instantiation patterns in human semiosis.

New Times for Multimodality? Confronting the Accountability Culture

AbstractAs new times become hard times, there may be little time for multimodality in school unless educators confront the accountability culture. This commentary reviews the arguments for multimodal transformations of school literacy curricula and explores the potential of reflective talk about multimodal meaning‐making as an assessment practice. Talking about how multimodality works may enable educators and students alike to talk back to an accountability culture that limits what counts as literacy.

Accessing Pedagogical Territories That Can't Be Put into Words: Using Video to Build Understandings of Children's Multimodal Meaning-Making

This article outlines ways in which video can further our understanding of how different modes of communication and meaning-making shape learning and learners in the early years. It focuses on a dramatic play and writing episode videoed during a three-year action research study investigating children's use of different semiotic modes in the curriculum of a New Zealand kindergarten. It highlights the capacity of video to enable closer attention to be paid to the pedagogical significance of modes such as the visual, gestural, mimetic, spatial and kinaesthetic as well as the verbal. It explores how differences in media (e.g. computer, video, book, screen) interact with differences in mode, and the ways in which the collaborative viewing of video recordings of ‘everyday’ episodes in early childhood settings, by teachers, researchers and parents, can serve as a platform for inquiry about children's meaning-making processes.