Digital Multimodality Research Articles

Multimodal immersion in English language learning in higher education: A systematic review

The development of multimodality has led to increased research on its use to improve students' English language competency. However, no recent review has analyzed multimodality in English language learning in higher education. This systematic review examines 34 research articles published from 2013 to 2024. The primary focus of the study is to explore the application of multimodal pedagogies in higher education, the methods and materials used to assist learners in acquiring English language skills, the English language skills acquired through the usage of multimodality, and the main results of using multiple modes. This systematic review employs the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) standards. It adopts a thorough search strategy across electronic databases, which include Web of Science and Scopus. We found (1) the implementation of multimodality contributes to learners' English language proficiency in English for academic purposes (EAP) and English for specific purposes (ESP) education; (2) there is a predominance of digital multimodality and nonverbal communication use in the higher education classroom, for example, gesture, kinesics, spatial position, facial expression, and gaze the use of the 3D (dimensional) environment, virtual reality (VR); (3) there is an advantage of a multimodal approach in improving higher education learners' vocabulary, reading, speaking, and writing skills and a positive connection between the implementation of multimodality and the development of learners' communicative ability. This systematic review highlights existing research gaps and outlines potential avenues for future investigation aimed at conceptualizing and assessing learners’ skills through multimodal approaches.

Digital Synesthesia in Heritage and Second Language Writing during Collaborative and Individual Digital Storytelling

Using the social semiotic multimodal approach, this study compared the transformation and transduction moves of eight Spanish heritage language learners (SHLs) and six second-language learners (L2s) as they composed collaborative and individual digital stories (DSs) in an undergraduate advanced Spanish writing class. The research analyzed the learners’ composing processes regarding digital synesthesia when working collaboratively and individually as they integrated written, oral, aural, and visual semiotic resources. The data revealed that SHLs demonstrated more frequent and more complex digital synesthesia than did their L2 classmates during their quest for digital multimodality in Spanish, indicating improvement via task repetition in frequency and variety of integrated modes.

Google doodles as multimodal storytelling

This article highlights increasing multimodality of online communication, in particular, multimodal alteration of Google logo known as Google Doodle. Doodles are interesting in terms of multimodal communication, semiotic landscape, encoding and decoding information, meaning-making, visual storytelling and visual communication in the globalized world, which remains multilingual, multicultural, multiracial, multiethnic and prone to conflicts. Despite the problems of aberrant decoding Umberto Eco described, visual communication proves to be the economical and effective means of informing the global audience. Google Doodles progressed from simple pictorial elements added to the Google verbal logo to intricate images with complicated multi-tier historical and cultural background, animated, interactive, hyper-linked, AI-based, video doodles, puzzles and even diverse games. Doodles followed and mirrored ICT and visual design evolution, developed in parallel with digital design and digital multimodality. Google used the chance to implement and demonstrate digital design potential in modern communication. Google Doodles became bright examples of infotainment, edutainment and gamification as every doodle was to inform, to educate, to astonish, to trigger WOW effect while celebrating some remarkable personality or event. Every Doodle is the result of thorough research into the issue, of sophisticated visual and digital design, of unique imaginativeness and high creativeness of the world best mind what makes it possible to squeeze an elaborate story into a picture or a short video. This study suggests that Google Doodles tend to become unique samples of multimodal storytelling, highly complicated semiotically and cognitively. Doodlers make great fun for many people, they send them googling to track down amazing stories, to discover intriguing details, and broaden knowledge horizon. Support of Doodle 4 Google competition among kids promotes multiliteracy and digital culture what is crucial to keep young generations future-proof.

ЦИФРОВА МУЛЬТИМОДАЛЬНІСТЬ У ДИСКУРСІ МУЗЕЇВ ПІД ЧАС ПАНДЕМІЇ COVID-19

COVID-19 pandemic triggered radical changes and transformations in modern world, business, professional fields, communication including. In the wake of lockdowns and quarantines museums had taken a hit too. Museums had to remain accessible and visible, at least in the digital domain, therefore they had to shift to virtual realm, to offer digital services, to increase their activity in social media. Museums faced a number of challenges like dealing with global digital audience, cognitive modelling of the online content and online user. Their efficiency in the digital domain was different, so museums evidenced 10 – 200% increase of online visits and growth of global digital audience. Due to implementation of digital technologies small local museums made a breakthrough to multimodal domain and received international recognition. In pre-pandemic world, museums had been working on their virtual environment, opened virtual-real museums and extended multimodality of exhibitions, making visitors impressions multisensory. During the pandemic virtual exhibitions, virtual tours, films, education material have become more multimodal and interactive. In films and virtual tours the focus was on stories not on 3D image of the exhibits and a standard 360 tour only. They experimented with modes and conquered new media, used multimodal storytelling. Museums tried to create a sense of presence and place and to relocate online visitors from their homes to museums. To encourage children, museums implemented gamification and edutainment strategies, created virtual games. Museums started a new trend – collecting online (collecting COVID-19, BLM), creating Big Data and digital collections which might be used for multi-vector multidisciplinary research. Digital multimodality might become an indispensable component of museum online landscape in the post-pandemic world.

Designing the New Negro: The Color of Late Nineteenth-Century Multimodality

Abstract Focusing on the 1900 publication of A New Negro for a New Century, this article examines multimodal design's “social and semiotic history” ( Kress, 2010 , p. 46), arguing that multimodality as semiotic potential and design as semiotic performance were both racially charged in the late nineteenth-century United States. By positioning multimodality, and thus designing writing, historically, this essay exposes the racial coding of design. It reveals the challenges posed by multimodal agency for African Americans as well as their concerted efforts to claim multimodality as a rhetorical resource serving the goal of racial uplift. To underscore the color of fin de siecle multimodal design, the article begins by addressing the racial valance of the book as medium, situating A New Negro for a New Century as a communication event within post-Reconstruction print culture, a transitional moment for technology and publication processes. Then, the article turns to the two key modes of image and word, contending that the White dominant culture sought to construe image and word as White resources. At the same time, the African American community fought back by seizing and exercising their modal agency despite overt and covert resistance. This analysis emphasizes that, as a semiotic performance, late nineteenth-century multimodality was neither neutral nor colorless. Instead, design was interpolated with race, suggesting the need to examine digital multimodality through a racial lens. Thus, the article concludes with a three-part heuristic by which the racial dimension of digital multimodality might be made visible.

Design and Implementation of a Digital Multimode H∞ Controller for the Spey Turbofan Engine

In this paper, a 2 degrees-of-freedom multimode controller design for the Rolls Royce Spey turbofan engine is presented. The controller is designed via discrete time H∞-optimization; it provides robust stability against coprime factor uncertainty, and a degree of robust performance in the sense of making the closed-loop system match a prespecified reference model. Multimode control logic is developed to preserve structural integrity of the engine by limiting engine variables to specified safe values. A simple strategy for antiwindup and bumpless transfer between controllers, based on the Hanus anti-windup scheme (1987, “Conditioning Technique, A General Anti-Windup and Bumpless Transfer Method,” Automatica, 23(6), pp. 729–739) and the observer-based structure of the controller, is presented. The structure of the overall switched controller is described. Actual engine test results using the Spey engine test facility at Pyestock are presented. The controller is shown to perform a variety of tasks, its multimode operation is illustrated and improvements offered on existing engine control systems are discussed.

Research Directions: Mobile Texts and Migrant Audiences: Rethinking Literacy and Assessment in a New Media Age

This article highlights the importance of rethinking literacy assessment in a digital and global world. Although pressures currently abound to narrow conceptions and practices of literacy, especially in an era of high stakes testing, digital multimodality and connectivity offer the potential for new ways of thinking, representing, and communicating, as well as new avenues for participating in relationships across social, geographic, and cultural difference. We explore the challenges of redesigning assessments so that they better take into account children’s multiliterate capabilities. In so doing, we offer examples from our work in afterschool contexts that demonstrate how we have grappled with the complexities of assessment in new times.

Equalization of Multimode Optical Fiber Systems

Decision feedback equalization (DFE) may provide a simple way to dramatically increase the bit rate on multimode optical systems. Computer simulations of digital multimode systems with tapped delay-line linear equalization and/or DFE are described. Results indicate that with relatively simple hardware it may be possible to increase by many times the usable bit rate on dispersion-limited optical channels, with a power penalty less than that for multilevel transmission and without the latter's complexity. For example, one could double the bit rate of a two-level system by paying an equalization power penalty of about 3 dB with DFE, whereas doubling the bit rate by going to four levels requires a penalty of 4.8 dB.