Multimodal Composition Research Articles

Graphic Novels for Multimodal Learning: Equity for English Learners

This article provides a synthesis of current research and theory that makes a case for using graphic novels to improve adolescent literacy achievement through multimodal, equitable learning opportunities for English learners (ELs) while meeting the required English Learner Proficiency Standards (ELPS). Supported by multimodal theory and culturally responsive teaching (CRT) approaches, research shows that graphic novels are inspirational teaching tools for adolescent ELs to engage more deeply with reading and writing. Specific recommendations for classroom use of graphic novels include a focus on a) motivation and engagement, b) discussions, c) visual information, and d) multimodal composition. Educators can make use of strategic instructional practices with graphic novels to increase overall literacy success for ELs and adolescent learners in general.

Research: An Interconnected Framework for Assessment of Digital Multimodal Composition

Drawing from the Beliefs for Integrating Technology into the English Language Arts Classroom, as well as prior scholarship on digitally mediated communication, rhetorical studies and composition, assessment, and digital literacies, this theoretical article presents a framework for creating and assessing digital multimodal compositions. The Interconnected Framework for Assessment of Digital Multimodal Composition conceptualizes digital multimodal composing through three interconnected and layered domains: audience, mode and meaning, and originality. Though the three domains are defined individually, they are inextricably linked within the recursive processes and products of digital multimodal composing to contribute to intended meaning. The authors describe and justify the domains, present assessment considerations, and conclude with implications for practice and suggestions for designing assessments relevant to context and task.

Encountering unnatural E-literature: tracing interpretation and relationality across multimodal response and digital annotation

ABSTRACT The emergence of e-literature – texts created on and for digital devices – has coincided with innovative, transgressive methods of storytelling. Pry, an iOS-based text, exemplifies e-literature’s potential to rethink traditional narrative conventions. Rather than depict the real-world as we experience it (i.e. mimesis), Pry features metaleptic elements (i.e. jarring transgressions across narrative levels) and unnatural temporality (i.e. nonlinear, contradictory jumps across time). This article traces how a graduate class of librarians and preservice teachers responded to Pry’s unnatural narratology through multimodal composition and annotation. Findings suggest that e-literature may demand more expansive repertoires of text interpretation and relationality.

Scaffolding Multimodal Composition With Emergent Bilingual Students

Digital multimodal composition offers opportunities for emergent bilingual (EB) students to orchestrate semiotic resources in ways that develop their identities, strengthen their understandings of language, and help them to engage with content. To better understand how EBs can participate in varied multimodal composing practices, this study systematically reviews the literature on EBs’ digital multimodal composing in secondary classrooms. More specifically, it examines types of scaffolds, or planned and responsive instructional supports, used by teachers and students, as well as functions for learning associated with these scaffolds. Through an inductive approach, the authors analyzed 74 studies situated in classrooms. Findings showed seven types of scaffolding: collaboration, direct instruction, exemplar texts, translanguaging, discussion, encouragement, and questioning. In addition, eight scaffolding functions emerged that illustrate three major themes of scaffolding identities, scaffolding resources, and scaffolding contexts. The authors then discuss implications for classroom practice, implications for translanguaging and social semiotics theories, and directions for future research.

Painting Is Poetry: Multimodal Composition and Untold Stories in The Odyssey

Preview this article: Painting Is Poetry: Multimodal Composition and Untold Stories in The Odyssey, Page 1 of 1 < Previous page | Next page > /docserver/preview/fulltext/ej/110/5/englishjournal31225-1.gif

Korean Primary School Students’ Experiences of Multimodal Composition in English Classes

Korean Primary School Students’ Experiences of Multimodal Composition in English Classes

Rewriting deficit storylines: the positioning of one fourth-grader as comics expert

PurposeThis study took place in an elementary English language arts classroom during a comics writers workshop unit and focused on one fourth-grade author. This paper aims to explain how a fourth-grade student receiving special education services positions himself and is positioned by others as an expert during a unit on comics. When students’ knowledge about multimodal composition is recognized and valued, the classroom community can become a place where students can author themselves into positions of power and authority.Design/methodology/approachInformed by sociocultural theory, social semiotics and positioning theory, the authors conducted a qualitative study to analyze the focal participant’s published comic, classroom interactions and interview data using open and in vivo coding systems.FindingsThe findings documented how the focal participant was positioned as an expert by others and how he positioned himself as an expert. The findings also explore how this fourth-grade comics expert left an authorial residue that extended beyond the boundaries of this particular comics unit, impacting his teachers and future iterations of the comics workshop.Originality/valueScholars have theorized multimodal approaches to reading and writing pedagogy as an equitable enterprise that values meaning-makers using a wide variety of semiotic resources. This study shows how incorporating an explicit multimodal composition opportunity allowed one fourth-grade author space to craft a comic and re-author traditional classroom positions of power.

Painting and Podcasting as Play: Assemblages (With)in Classroom-Situated Multimodal Design

BackgroundResearchers interested in more playful literacy practices often abandon school-related contexts to instead follow young people to after-school settings, community spaces, social spaces, home environments, and online hangouts. However, play also makes itself visible, even (and sometimes especially) in more formalized learning spaces. Sometimes it emerges subversively in much the same way that flowers grow through cracks in concrete. Other times, teachers deliberately carve out experiences for students to make space for humor, creativity, laughter, and social imagining.Purpose/Objective/Research QuestionThis inquiry examines two questions: What assemblages of play emerge when youth engage in multimodal composition in classroom spaces? What did these assemblages of play produce?ParticipantsParticipants included youth and teachers from two very different classrooms engaging in a diverse set of multimodal composition projects: one 6th-grade class in an independent school engaging in the creation of podcasts, and one 8th-grade class in a rural public school painting a scene from a recently read novel.Research DesignThis inquiry was designed as a qualitative case study.Data CollectionData sources gathered for this project included descriptive fieldnotes taken during approximately 3 hours of classroom observation during multimodal design, audio recordings during class sessions, photographs taken at the school sites, digital products produced by youth, reflective interviews with teachers, follow-up focus groups with 6th-graders, and reflective surveys filled out by all 8th-grade participants.FindingsTwo focal assemblages emerged from various participant/researcher vantage points. Youth conceptualized play as craft, and the researcher observed work-play flows.ConclusionsRecommendations include (a) better recognizing play across age groups and places/spaces, (b) conceptualizing play as a complicated assemblage of materials, humans, emotions, technologies, and space-times, and (c) embracing work-play flows.

“You’re the boss, yo!”: Role-Play in Digital Multimodal Composition of Newcomer Youth

This case study explores how two 16-year-old newcomer youth in a Canadian secondary school navigated the digital multimodal composition process through role-play in their first digital video production at school. Employing a qualitative, case-study design, the study shows how the youth playfully accentuated collaborative over coercive power relations, as well as repositioned and represented their imagined identities as they played different assigned roles in the filmmaking process. The implications of these findings are discussed for educators and researchers considering digital multimodal composition as a classroom literacy practice.

Student perspectives on multimodal composing in the L2 classroom: tensions with audience, media, learning and sharing

PurposeMultimodal composing is often romanticized as a flexible approach suitable for all learners. There is a lack of research that critically examines students’ perspectives and the constraints of multimodal composing across academic contexts. This study aims to address this need by exploring high school learners’ perspectives and experiences enacting multimodal learning in an L2 classroom. More specifically, this study presents key tensions between students’ experiences of multimodal composing and teacher/researchers’ use of multimodal composition in an L2 classroom setting.Design/methodology/approachThe paper focuses on two multimodal composing projects developed within a design-based implementation research approach and implemented in a high school French class. Multiple data sources were used: observations; interviews; written reflections; and multimodal compositions. Data were analyzed using the critical incident technique (CIT). A critical incident is one that is unplanned and that stimulates reflection on teaching and learning. Methodologically, CIT was enacted through iterative coding to identify critical incidents and collaborative analysis.FindingsUsing illustrative examples from multiple data sources, this study discusses four tensions between students’ experiences of multimodal composing and teacher/researchers’ use of multimodal composition in a classroom setting: the primary audience of student projects, the media leveraged in student projects, expectations of learning in school and the role of a public viewing of student work.Originality/valueThis paper problematizes basic assumptions and benefits of multimodal composing and offers ideas on how to re-center multimodal composing on student voices.

Cinematic Reproduction as Multimodal Composition in First-Year Composition

This case study introduces an assignment from a large midwestern university FYC program, which emphasizes analytical writing by introducing students to a series of increasingly layered texts, including film. This has many advantages: teaching multi-perspective, evidence-based visual and textual analysis. For students to see textual constructedness the author required they remake film scenes using their cellphones, employing and then articulating their use of generic conventions. This article details the original assignment and tracks its evolution due to students’ innovations. One group’s project, a scene from The Shining, highlights multimodal recreation’s capacity to help students re-envision and more fully analyze cinematic elements, adding depth and specificity to students’ analytical writing.

Fostering Pre-Service Teachers’ Critical Multilingual Language Awareness: Use of Multimodal Compositions to Confront Hegemonic Language Ideologies

ABSTRACT Drawing upon tenets of Critical Multilingual Language Awareness (CMLA), we analyzed multimodal compositions created by preservice teachers (PSTs) from two institutions to investigate their meaning-making at the nexus of language, identity, and power. Through analysis of PSTs’ multimodal compositions, reflective writing about their coursework, and retrospective interviews, participants shared how these compositions informed their working understandings of CMLA in relation to monoglossic and raciolinguistic language ideologies. Findings from our study suggest that through multimodal composition, PSTs demonstrated varied levels of awareness that one’s language practices determine whether they experience acceptance, belonging, and access to education; some PSTs also acknowledged that teachers serve as adjudicators of language, who may reinforce or reject hegemonic language ideologies. This study reinforces the efficacy of integrating opportunities for PSTs to represent their learning through multimodal composition to foster their CMLA.

The uses of writing for digital video

ABSTRACT In this essay I view writing as a communicative practice, and ask how and in what ways teachers and students define the role of writing in relation to digital video composition. While there is much research on the importance of multimodal composition, and teachers are being encouraged to incorporate more digital making into their curricula, we need to more fully examine children’s classroom writing and meaning-making practices as they work in newer modes like video. I use a sketch of the work of one student to examine what I see as the continued primacy of paper-and-pencil or computer-screen writing even in a digital video making process. I also discuss the ways children saw writing along a continuum of utility, from more to less useful, depending on their own perspectives on writing in general and on writing’s role in the production of videos. I show how the boundaries that teachers maintain between writing and speaking matter less to some students than they do to teachers. I close with some suggestions for how we should approach the theoretical and pragmatic challenges of writing for digital video production in school.

Multimodal composition pedagogy in Higher Education: a paradigm shift

The use of digital technologies for pedagogical purposes worldwide has augmented the need for radical and urgent changes in academic literacy education at tertiary institutions. In this article, multimodal composition alternatives to traditional written academic assessments are proposed as a platform for transformation in academic literacy modules. The literature framework highlights the hindrances and affordances of multimodal composition pedagogy. A qualitative research approach was followed in the empirical research. An online questionnaire was used in this interpretivist research design. The opinions of academic literacy lecturers at a South African university were sought to gauge how they felt about incorporating alternative multimodal assessments instead of traditional methods such as text-based compositions only. The inclusion of student voice and agency regarding multimodal academic literacy pedagogy was also investigated. The findings revealed that many lecturers did incorporate formative multimodal teaching and learning strategies to scaffold academic writing. However, with regards to summative assessment of academic literacy skills, most lecturers still preferred the traditional essay. Most of the lecturers were also not in favour of student contribution to multimodal content and assessment. It is the responsibility of all stakeholders to ensure that multimodal composition transformation is implemented to support students’ academic literacy needs, not their own, in an ever-changing digital landscape in higher education.

Searching for Street's "Mix" of Literacies through Composing Video: Conceptions of Literacy and Moments of Transfer in Basic Writing

This paper examines three students’ multimodal composition experiences in Basic Writing where conceptions of literacy interacted with moments related to transfer across media. Extending Brian V. Street’s work on literacy and Rebecca S. Nowacek’s transfer theory to multimodal composition through video, we use analysis of ethnographic data to conclude that for some students, video facilitated both a robust conception of literacy as ideological and transfer across media. For others, external forces inhibited opportunities for transfer and reinforced a conception of literacy as autonomous. We close reflecting on how we might more usefully scaffold student learning for transfer and more complex conceptions of literacy.

From the Editors

From the Editors

Digitalization and the Writing Classroom: A Reflection on Clasroom Practices

This paper outlines the educational benefits of creating digital stories for a variety of academic purposes as well as the professional need for students to develop and showcase digital proficiency. Digital stories fall under the category of multimodal composition and new media studies, and they encourage students to expand their digital literacy skills while reconceptualizing ways in which traditional writing projects can appeal to a broader audience. The article also addresses some of the classroom challenges teachers may face when trying to implement the practice and some practical resources that might assist teachers to integrate digital stories into their classrooms.

The experiential nature of ELF reformulations in the multimodal representations of modern and ancient sea-odysseys

This chapter explores the latest stage of a research project carried out at the University of Salento, which aims to investigate the effects of emotions on the positive reconsideration of mass migrations and intercultural socialization. In particular, this study illustrates the linguistic and multimodal strategies of production of four videos that are informed by an unbiased discursive frame for the multimedia representations of sea-journeys. In these clips, the linguistic and audiovisual associations between samples of Western and Non-Western migration narratives are meant to assist viewers in acknowledging that travelers from past and present times experience shared emotions and feelings when leaving their native countries in search for better life conditions. The activation of this perlocutionary effect is pursued by means of hybridizations between written and oral accounts of sea-voyages and between ‘epic movie’, ‘mockumentary’ and ‘journalistic interview’ genres. Narrations are retextualized into ‘experiential reformulations’ that resort to modern variations of English as a Lingua Franca (ELF) in order to underline, through their structural and phonetic characteristics, the dramatic and life-threatening nature of migrations. The audiovisual composition mixes factual and cinematic properties so as to attract envisaged viewers’ attention and then induce, in them, the positive reconsideration of the objects of representation. After detailing the main phases of experiential reformulation and illustrating the multimodal composition of the four videos, this chapter comments on the results of a reception study of the multimedia research products, in order to enquire into the empirical reception of genre and text hybridization, as well as into the accessibility level of the ELF variations that represent the verbal dimension of the videos.

Segmentation of Brain Magnetic Resonance Images using Deep Learning Classification and Multi-modal Composition

Background: Accurate detection of brain tumor and its severity is a challenging task in the medical field. So there is a need for developing brain tumor detecting algorithms and it is an emerging one for diagnosis, planning the treatment and outcome evaluation. Materials and Methods: Brain tumor segmentation method using deep learning classification and multi-modal composition has been developed using the deep convolutional neural networks. The different modalities of MRI such as T1, flair, T1C and T2 are given as input for the proposed method. The MR images from the different modalities are used in proportion to the information contents in the particular modality. The weights for the different modalities are calculated blockwise and the standard deviation of the block is taken as a proxy for the information content of the block. Then the convolution is performed between the input image of the T1, flair, T1C and T2 MR images and corresponding to the weight of the T1, flair, T1C, and T2 images. The convolution is summed between the different modalities of the MR images and its corresponding weight of the different modalities of the MR images to obtain a new composite image which is given as an input image to the deep convolutional neural network. The deep convolutional neural network performs segmentation through the different layers of CNN and different filter operations are performed in each layer to obtain the enhanced classification and segmented spatial consistency results. The analysis of the proposed method shows that the discriminatory information from the different modalities is effectively combined to increase the overall accuracy of segmentation. Results: The proposed deep convolutional neural network for brain tumor segmentation method has been analysed by using the Brain Tumor Segmentation Challenge 2013 database (BRATS 2013). The complete, core and enhancing regions are validated with Dice Similarity Coefficient and Jaccard similarity index metric for the Challenge, Leaderboard, and Synthetic data set. To evaluate the classification rates, the metrics such as accuracy, precision, sensitivity, specificity, under-segmentation, incorrect segmentation and over segmentation also evaluated and compared with the existing methods. Experimental results exhibit a higher degree of precision in the segmentation compared to existing methods. Conclusion: In this work, deep convolution neural network with different modalities of MR image are used to detect the brain tumor. The new input image was created by convoluting the input image of the different modalities and their weights. The weights are determined using the standard deviation of the block. Segmentation accuracy is high with efficient appearance and spatial consistency. The assessment of segmented images is completely evaluated by using well-established metrics. In future, the proposed method will be considered and evaluated with other databases and the segmentation accuracy results should be analysed with the presence of different kind of noises.

Embodied Subjectivities and the City: Intervening in Local Public Debates through Multimodality

This article describes and reflects on a place-based pedagogical approach to public engagement that uses multimodal composition to insert new discourses into ongoing local debates over university expansion. The public-forming potential of multimodal texts encourages students to imagine new ways of being public and opportunities for adopting community-oriented subjectivities that engage with the issues, people, and spaces in neighborhoods adjacent to campus.