Multimodal Genre Research Articles

INFOGRÁFICO COMO GÊNERO MULTIMODAL MOTIVADOR: PROPOSTA DE ENSINO DE LEITURA E ESCRITA

This text presents the result of a study on the challenges of teaching reading and text production in the Portuguese language discipline in elementary school. It deals with a research developed at PROFLETRAS (Postgraduate Program in Language Course / Dourados/MS) with the purpose of knowing the multimodal infographic genre as a strategy for reading and producing opinion texts. In the intervention we use the use of ICTs - Information and Communication Technologies and the teaching / learning practices of the Portuguese language, given the students' lack of interest in reading linear and monomodal texts. In this way, the activities carried out were based on the use of a virtual environment, from surveys, readings of infographics and the production of opinion texts. The methodology of the study was that of action research, with the involvement of students in activities related to multimodal genres, specifically, the infographic genre. This textual genre had the purpose of subsidizing students in the production of opinion articles respecting the degree of difficulty in relation to the eighth year of Elementary Education and the creation of an online magazine, as a product. Theorists such as Rojo (2016), Kress and Van Leeuwen (2001,2006), and Ribeiro (2016), among others, subsidized the studies. The development of this multimodal intervention and the performance of the proposed activities brought to school a teaching method different from the usual one and students connected with innovations and the use of new technologies. This proposal made students aware of the knowledge of the multimodal infographic genre and provided skills for the process of reading and writing as a reading subject and an opinion maker, becoming critical individuals in society. Keywords: Literacy practice; Reading and writing; Infographic.

Video Scenes Segmentation Based on Multimodal Genre Prediction

Abstract Recent technologies’ understanding videos content remain limited due to its complexity and length. However, videos segmentation into small coherent units facilitates indexing and searching task. The subjectivity remains the essential constraint of videos, but the genre (drama, action...) does not present any conflict. In this paper, we present a new approach to video segmentation into scenes based on genre prediction. Initially, the video is divided into shots of equal duration. We used architecture, based on audio-visuals deep features extracted from trained neural networks for genre prediction, and we introduced a transition detection method based on the similarity calculation between shots genre. The originality of this method consists in using the highly level semantic relationship between successive shots for transition detection. We reached good performances on videos of the multi varied genre. We used the RAI dataset and BBC dataset to evaluate our method through a comparison with other state-of-the-art approaches.

Research Video Abstracts in the Making: A Revised Move Analysis

This article reports on a revised move analysis of a video abstract (VA) repository curated by Cell Press. The analysis reveals that the VA displays several distinct core moves, optional moves, and move units. The analysis suggests that albeit evidence being inconclusive due to the sample size, the VA is in effect an emerging genre whose implications for further research and practice are worth discussing. The article also questions the assumed analytic pertinence of move analysis to researching multimodal genres and cautions against the uncritical use of the VAs curated by leading academic publishers and journals in professional training.

“See Figure 1”: Visual moves in discrete mathematics research articles

This research is motivated by John Swales' genre analysis of academic texts and rhetorical studies of visuals in scientific texts. Swales' approach still generates insights into the rhetorical structure of research genres across disciplines, yet few studies explore their nonverbal aspects. Rhetorical studies of visuals examine images and ignore links with surrounding texts and may overlook ways that the nonverbal contributes to the discipline's intellectual project. ESP genre research has begun multimodal analysis of academic genres but its focus on conference presentations has left a gap in the study of verbal-visual links in other academic genres. This paper addresses this gap in multimodal genre scholarship. After analyzing a corpus of 30 RAs from discrete mathematics to identify the functions of visuals and possible visual–move associations, we found that visuals in two-thirds of the corpus perform three functions: ontological, argumentative, and epistemological. Our data also indicates three multimodal rhetorical moves initiated primarily by visuals. Implications of our findings are that visual moves go beyond textual considerations, they can disrupt the RA's chronological structure, and novice writers in the field would benefit from understanding the crucial associations between the visual representations, disciplinary knowledge, and the rhetorical structure of RAs in this field.

Digital memes and US pop politics. Dynamism and pervasiveness of a digital genre in the mobile Web era

Abstract – This paper analyses US political memes relating mainly to the Obama and Trump presidencies as an illustration of a subgenre shaped by today’s mobile culture, and thus part of the rise of new forms of digital textuality. In particular, the study explores the connection between technology, participatory culture and the entrenchment of memes as a multimodal genre within political discourse. The illustration of the diffusion of this political meme subgenre as a grass roots phenomenon, generated and distributed through social media, is illuminated by the article’s focus on the often ironic comments it makes about US politics. By reconstructing the evolution of digital memes in this period and illustrating and exemplifying the textual processes that give this political subgenre its unique forcefulness, the article concludes that digital memes have embraced textual forms – based on mass media and shared authorship – that are certainly informal and often offensive, but which also testify to a vision of politics which is interpretable within the wider framework of US popular culture, the American Dream in particular.

Vine Racial Comedy as Anti‐Hegemonic Humor: Linguistic Performance and Generic Innovation

This paper analyzes the semiotic features and sociopolitical commentary of anti‐hegemonic Vine racial comedy in videos of Viner King Bach. The following analysis demonstrates that Vine racial comedy is heavily influenced by the generic features of African American stand‐up and sketch comedians and existing multimodal genres of online discourse, but King Bach's adaptation of these features to the unique affordances of the Vine platform situates Vine racial comedy beyond the boundaries of the genres it draws on. Analysis of this digitally mediated multimodal discourse demonstrates that, while not deterministic, the medium through which discourse is constructed can significantly impact how a discourse genre is conceptually understood by expanding the semiotic resources through which the sociopolitical and discursive goals that define a genre are accomplished.

Discourse and Representation in Emblematics: Hermeneutic and Ideological Implications of Stylistic and Cognitive Analyses

The article focuses on the dialectic relationship between visual and verbal representations in Renaissance emblematics, a multimodal genre in which words and images were inherently interactive and physiologically intermingled. Devices and emblems were

EXPERIENCE OF DEVELOPING STUDENTS’ MULTIMODAL LITERACY IN THE DIGITAL LEARNING ENVIRONMENT OF HIGHER EDUCATION INSTITUTIONS

The paper presents a theoretical overview of the notion “multimodal literacy” in the context of the multiliteracies pedagogy focusing on the research relevant to developing notions of literacy for twenty-first century higher education standards. The relevance of the topic is explained by the fact that the modern world is becoming more and more pluralist and increasingly dependent on new technologies and different modes of communication, students require new transdisciplinary skills and methods of learning. The author examines the changing pattern of literacy as broad, multidimensional knowledge, skills and attitudes. Literacy is viewed as a personal sociocultural capacity the development of which provides every student with the rules how to solve problems, to relate to others and survive in diverse social settings, and creates numerous opportunities for enhancing critical thinking and reflection as well. The foundation for students’ multimodal literacy in the digital learning environment of a higher education institution through theoretical and applied frameworks of visual, media, information and multicultural literacy is considered. The diverse affordance of digital tools and range of multimodal genres offer engaging new avenues for students’ creative expression on the basis of developing multimodal literacy. It is emphasized that successful life in a world of multimodal information requires fluency in a broad range of literacies to consume and create texts in visual, audio, and written formats, to evaluate messages in a variety of mediums, and to gain sociocultural awareness and the ability to communicate and live in a global diverse society. Pedagogic techniques across disciplines (multimodal essay, digital mapping project) are highlighted in view of their effectiveness in developing multicultural literacy of undergraduate students. It is an empowering practice for modern educators to incorporate multimodal tasks into a diverse learning environment, which teaches critical thinking and multimodal skills, enhances audio-visual rhetoric, develops writing and oral communication skills, and aids in self-expression and self-reflection.

Мультимодальность в сетевом политическом дискурсе: интернет-мемы о независимости Шотландии (Multimodality in Online Political Discourse: Internet Memes about Scottish Independence)

Russian Abstract: Статья посвящена медиатизации возможной независимости Шотландии в социальных сетях. Проводится мультимодальный анализ тематических интернет-мемов 2014-2016 гг. как образцов сетевого политического дискурса. Сетевой политический дискурс охарактеризован как объект лингвистического и мультимодального анализа. Этот вид дискурса рассматривается как неинституциональная разновидность политического дискурса вообще, отличающаяся неофициальностью, стихийностью, высокой скоростью реагирования на общественные вызовы и многоголосием. Отмечается способность сетевого политического дискурса к компрессии информации и оценочной интерпретации отражаемой действительности. Рассматривается понятие интердискурсивности как одной из жанрообразующих черт сетевого политического дискурса в целом и интернет-мемов в частности. Приводится определение мультимодального текста как задействующего вербальный и зрительный каналы восприятия, оба из которых содержат ключевые элементы информации для декодирования смысла. Рассматриваются различные определения интернет-мема как мультимодального жанра интернет-коммуникации. Выполнен анализ четырех интернет-мемов о независимости Шотландии. Мемы анализируются на предмет соотношения вербального и визуального компонента; стилистических характеристик текстовой составляющей; сильной и слабой позиции визуальной составляющей; источников прецедентности и интердискурсивного характера отношений между составляющими. English Abstract: The article is dedicated to the mediation of the possible independence of Scotland in social networks. A multimodal analysis of thematic Internet memes 2014—2016 is carried out as examples of online political discourse. Online political discourse is characterized as an object of linguistic and multimodal analysis. This type of discourse is considered as an uninstitutional kind of political discourse in general, characterized by informality, spontaneity, high speed of response to public challenges and polyphony. The ability of network political discourse to compress information and the estimated interpretation of reflected reality is noted. The concept of interdiscursiveness is considered as one of the genre-forming features of network political discourse in general and Internet memes in particular. The definition of multimodal text as involving the verbal and visual channels of perception, both of which contain key elements of information for decoding the meaning, is given. Various definitions of the Internet meme as a multimodal genre of Internet communication are considered. The analysis of four Internet memes about the independence of Scotland is given. Memes are analyzed for the relationship of verbal and visual component; stylistic characteristics of the text component; strong and weak position of the visual component; sources of precedent and interdiscursive nature of relations between components.

Мультимодальность в сетевом политическом дискурсе: интернет-мемы о независимости Шотландии

The article is dedicated to the mediation of the possible independence of Scotland in social networks. A multimodal analysis of thematic Internet memes 2014-2016 is carried out as examples of online political discourse. Online political discourse is characterized as an object of linguistic and multimodal analysis. This type of discourse is considered as an uninstitutional kind of political discourse in general, characterized by informality, spontaneity, high speed of response to public challenges and polyphony. The ability of network political discourse to compress information and the estimated interpretation of reflected reality is noted. The concept of inter-discursiveness is considered as one of the genre-forming features of network political discourse in general and Internet memes in particular. The definition of multimodal text as involving the verbal and visual channels of perception, both of which contain key elements of information for decoding the meaning, is given. Various definitions of the Internet meme as a multimodal genre of Internet communication are considered. The analysis of four Internet memes about the independence of Scotland is given. Memes are analyzed for the relationship of verbal and visual component; stylistic characteristics of the text component; strong and weak position of the visual component; sources of precedent and interdiscursive nature of relations between components.

Live Action Role Play: Transmediality, Narrativity and Markers of Subjectivity

This work frames larp as a multimodal medium: based on live-action role-playing but also open for inclusions of other media and technologies. It is approached as a narrative medium, used for performative and participatory cocreation of dynamic storyworlds. The emergent story is experienced by the full sensorium of the players’ bodies and simultaneously uses the bodies as a medium of representation. Players combine the roles of authors, actors, characters, and audience. In its prototypical model, larp strives for a high degree of perceptual unity between player and character and prefers iconic representation via live action and live acting, with lesser importance of indexical and symbolic signification. This study reaches for J.-N. Thon’s (2014, 2016) typology of forms of representation of character’s subjective perception in film, graphic novels, and video games, and applies it to live-action role-playing. The fundamental difference lies in the ‘firstperson audience’ (Sandberg 2004; Stenros & Montola 2010) mode of participation in larp. Unlike the audiovisual media discussed by Thon, in which it is intersubjective perception that comes unmarked, larp by default is experienced from a subjective spatial point of view. The first-person audience mode makes it hardly possible to switch to (quasi-)perceptual overlay, and to ever leave subjective spatial point of view, unless some of the prototypical principles of larp are broken. By contrast, representations of (quasi-)perceptual point of view and of imagined internal worlds are possible, although with some danger of unwanted metaleptic contact with another character’s delusions. For this and other reasons, larp has limitations in the use of simultaneous content markers and representational markers of subjectivity, whereas contextual content markers and all narratorial markers present no difficulty. The paper adds one category to Thon’s framework: symbolic markers, which are based on a pre-established code of arbitrary signs and are used to regulate the active agency or perception of the audience, e.g. a player’s gesture marking off-game status of the character, or movement from the main game area into a metaroom used for role-playing internal worlds. The overall discussion compares medium-specific affordances of live-action role-playing, which are fairly limited in representing subjectivity, to the affordances of larp as a multimodal genre, able to include many other forms of live, recorded and digitally mediated representation.

When context changes

Abstract This paper highlights the notion of dynamic context as an indispensable pragmatic aspect of multimodal argumentation and exemplifies a context-based approach to multimodal arguments with an analysis of the Mophie 2015 Super Bowl commercial. Whereas in dynamic semantics and verbal discourse analysis the notion of dynamic context and its context change potential are significant patterns for the analysis, argumentation theorists have not yet fully included these patterns in their discussions. The paper argues that multimodal argumentative genres such as commercials and movie trailers often work with a dynamically changing interpretation, which, at the end, reveals their persuasive patterns and final claims. It demonstrates that it is absolutely necessary for a detailed analysis of the argumentation in these multimodal genres to include the contextual influence and dynamic change potential. The paper thus emphasizes the need for an inclusion of the notion of dynamic context in methods and frameworks dealing with the complexity of multimodal argumentation.

Feature: Beyond Words on the Page: Using Multimodal Composing to Aid in the Transition to First-Year Writing

This article reports on a multimodal podcasting unit conducted during a two-week modified summer bridge program for at-risk incoming first-year students. The examples from student work show how teaching a multimodal genre encourages writers to draw from their prior knowledge of standardized genres learned in high school to effectively transition to college composition.

Distance crossing and alignment in online humanitarian discourse

This article analyzes multimodal genres of current online humanitarian discourse such as mission statements, annual reports and photo galleries to find how the construals of beneficiaries and humanitarian organizations align with the motives, values and emotional dispositions of prospective donors. The discursive reduction of distance between the donor and the beneficiary is likely to produce solicitation effects and enable self-legitimization. First, based on extant literature, the article develops a method to account for the pragmatic operations of textual ‘proximization’ and visually simulated ‘co-presence’ in humanitarian communication. Then it applies it to a sample of multimodal online messages issued by a prominent Polish humanitarian organization that distributes aid to communities in Africa or Asia. Analysis shows that Polish Humanitarian Action's mission statements and annual reports include strategic construals of space, quantity and transfer of aid that legitimize the organization's activities and their underlying axiological motivations. The texts also reproduce us/them differences to proximize the other that has been ‘Westernized.’ Meanwhile, the photo-galleries manufacture co-presence to enhance the axiological and affective investments in solidarity with the distant beneficiary. The case study offers a preliminary insight into alignment and distance-crossing maneuvers in online appeals that seek to project ‘proper distance’ between the donors and beneficiaries.

Practice-based perspectives on multimodal documents: Corpora vs connoisseurship

Multimodal studies are essentially interdisciplinary, crossing boundaries between the verbal and the visual, between creation and consumption, and, it is argued here, between academic analysis and professional practices. This paper presents a practice-based perspective on multimodal document genres which feature typography and page/screen layout. It is argued that in order to reliably account for observed data, analysts need to distinguish between effects which represent an effort to create meaning, those which are by-products of production technology, and those which are intended as navigational support for users. A range of problems is identified, facing those who seek to classify multimodal genres, including problems of method, interdisciplinarity, naming, granularity, and exemplification. Additional problems particular to the study of real documents include design competence, dysfunctional genres, creativity and continuous technical change. Insights from design practice are discussed, introducing a pattern language approach to develop a level of connoisseurship and good judgement that complements the more formal analysis represented by corpus studies and taxonomy.

Selfies in ‘mommyblogging’: An emerging visual genre

This article employs multimodal discourse analysis to explore how mothers represent their everyday experiences of motherhood on Instagram through different forms of self-portraiture. It investigates whether the ‘selfies’ that they share can be characterized as a visual genre and identifies four subgenres: presented, mirrored, inferred and implied selfies. The article illustrates the different ways in which the photographer’s perspective can be represented in each subgenre. The aim is to show that the function of the selfie as a multimodal genre is not solely to represent ‘the self’ but rather to enact intersubjectivity, that is, to generate various possibilities of relations between perspectives on a particular topic, issue, or experience and hence to open up potential for negotiating different points of view.

Da decodificação à produção de sentido:

Na perspectiva da Linguística de Texto e da Psicolinguística, a leitura adquire a categoria de processo incontestavelmente marcado pela elaboração/construção de sentido. Esse processo é viabilizado pela articulação de um amplo contingente de saberes prévios do leitor, bem como práticas cognitivas e metacognitivas projetadas pelos leitores, como: antecipações, inferências, paráfrases, seleções, entre outras (KLEIMAN, 1989; KOCH & ELIAS, 2006). Hoje, o ensino da leitura tem como meta efetivar a formação de leitores autônomos, competentes e proficientes. Isso envolve um trabalho didático ancorado nos gêneros do discurso. Entre estes, estão os gêneros multimodais. Estes são resultantes do entrelaçamento de distintos modos semióticos. Devido e essa propalação de semioses na sociedade pós-contemporânea, surgiu a necessidade da efetivação de um trabalho didático alicerçado na perspectiva dos multiletramentos. Isso transcende a perspectiva da decodificação e extração de informações. Este trabalho tem como objetivo refletir sobre a relevância da efetivação de um trabalho didático do ensino da leitura ancorado na perspectiva da multimodalidade discursiva e dos multiletramentos. Decorrente disso, analisamos seis exemplares de gêneros multimodais (Charges, Memes e Tirinhas), objetivando identificar as marcas multimodais que contribuem, para a efetivação do processo de atribuição de sentido.

Metadiscourse in Posters is Both Textual and Visual

Interest in genre analysis is far from declining. Just as information technology is moulding communication and how people communicate in the twenty-first century, scientific communities, academic members and other knowledge-dissemination means are also changing. Because academic and professional genres are not an exception, the impact of visual information on scientific communication is rendering multimodal analysis an increasingly necessary aspect for genre analysis to consider. This book responds to this new reality in the sense that it presents a multimodal genre analysis of academic poster presentations at conferences. The study aims to explore the textual and visual metadiscourse strategies used by poster presenters in order to make their research clear and to interact and engage with their readership. Due to the inexistence of reliable and representative data of the academic poster genre, a corpus of 120 posters from three disciplines (Law, Clinical Psychology and High Energy Particle Physics) had to be created by the author to find out whether there were any cross-disciplinary differences in the linguistic and visual metadiscursive resources of this rather under-researched genre.

Alfabetización como aprendizaje semiótico y discursivo: las producciones textuales de estudiantes en una escuela pública chilena

Literacy is directly related to meaning making and classroom activities allow students to learn how to use linguistic and semiotic resources to mean in a non-intuitive way. This research seeks to characterize from a semiotic and discursive view a sample of school textual productions, based on the theoretical frameworks of Social Semiotics and Systemic Functional Linguistics. The case study corresponds to a public school in 5th Region (Chile) and analyses an audiovisual corpus of curricular units of Natural Sciences and Social Sciences of a 3rd and 6th grade of Elementary School and a 1st grade of Secondary School. Through a Multimodal Discourse Analysis and genre analysis, the study reveals multimodal ways to define and explain in social and natural sciences as opportunities to expand the vernacular views of understanding the world, though in this case study they rely mostly on transcriptions from model texts. Keywords: Multimodal literacy, genres, semiosis.

Foreword

Click to increase image sizeClick to decrease image size Additional informationNotes on contributorsDenise NewfieldPROFESSOR DENISE NEWFIELD was an English teacher educator at the University of the Witwatersrand for many years and presently holds the position of honorary research associate. She is director of ZAPP, the South African Poetry Project. Her special interests are multimodality, multiliteracies, posthumanism, and poetry in education, particularly in contexts of disadvantage. Her recent publications include Multimodal approaches to research and pedagogy (with Arlene Archer) and ‘Reconceptualising poetry as a multimodal genre’ (with Raphael d’Abdon).