Elokuvan kuvailutulkkaus semioottisena kääntämisenä multimodaalisessa diskurssissa

Communication can be made by using two kinds of mode; they are verbal and visual communication modes. Nowadays, people tend to focus on the verbal communication and ignore the visual communication itself, even though they have the same important part of communication by using language. This study entitled “Verbal and Visual Communication in the Movie The Hunger Games” is trying to reveal and find out the sentence forms of the characters’ verbal utterances, the visual signs of the scene among the characters and the relationship between those communications. In the study, The Hunger Games movie is used as the data source. It tells about a death game that was held by a capitol city which was inhabited by people of the upper class and the people of the lower class who live in the districts as the tribute of the game. The method used to analyze the movie is Qualitative method. The main theory used is proposed by Dyer (1986) with his Verbal and Visual Communication theory and helped by using a theory from Timothy Shopen (2007) describing the types of sentence in form of verbal utterance. Based on the analysis, all of the sentence types as the verbal communication analysis and almost all of the visual sign elements in the movie were found, except for the national and racial element in appearance category. The verbal and visual communications represented in the movie supports each other to convey the meaning. It shows the differences between two different social classes; they are upper and lower classes. How the characters of the upper class speak, act, and look, tend to be more polite, prestigious, classy, colorful, and more educated than the characters of the lower class. It shows the relationship of visual and verbal communication itself.

Visual impairment is a consequence functional loss of vision which includes partially sighted low vision, legally blind and totally blind. To watch a movie is a challenge for the sight impaired people because most of the scenes are not verbally described in order to understand and have a full view of the program content. Due to this, it narrows down the opportunity of the sight impaired to access interactive learning in order to acquire knowledge and skills for independent learning. This paper presents the use of narration i.e., through spoken words and language in order to represent the visual content of chosen movie cartoons. To represent the visual content, an audio video description has been developed. Two cartoon movies were chosen. A user acceptance test was also conducted with two types of user groups. One group consists of 16 sighted students who were blindfolded to mimic the condition of visual impaired while the other group consists of 6 visual impaired adults. A set of questionnaire on the usability of the audio video description was distributed to the first users once they have completed reviewing the 2 movies. While the second group, was interviewed using the questionnaire as guide. Then, the 2 movies with audio descriptions were reviewed by 8 visually impaired children. Results have shown that the use of narration for audio video description can potentially be an avenue for the visually impaired to enjoy and experience a movie. The audio video description gradually can eliminate the dependence of the sight impaired individuals to their sighted companion who explains enough parts of the movie to them. Finally, the narrated audio video description will be another avenue for the sight impaired to access information through the use of multimedia.

Introduction: Audio description (AD) in museums is crucial for making them accessible for people with visual impairments. Nevertheless, there are limited museum-specific AD guidelines currently available. This research examines current varied international practitioner perspectives on museum AD, focusing on imagery, meaning, emotion and degrees of objectivity, and the regional differences (Europe and United States) in AD traditions, in order to better understand how AD can be used to enhance access to museums. Methods: Forty-two museum describers from 12 countries responded to a questionnaire requiring fixed-choice and free-text responses about the purpose and construction of museum AD. Results: Inference tests showed that European describers agreed more strongly than American describers that AD should “explore meaning” ( U = 91.00 , N1 = 24, N2 = 14, p = .03), and “create an emotional experience” ( U = 89.50, N1 = 24, N2 = 14, p = .03), rating the use of cognitive prompts as more important ( U = 85.50, N1 = 21, N2 = 14, p = .04). Qualitative data enriched this understanding by exploring participant responses on the themes of mental imagery, objectivity and interpretation and cognitive prompts. This highlighted broader agreement between regions on mental imagery, but more acceptance of interpretation in AD from the European respondents. Discussion: American and European describers’ opinions differ regarding the purpose of AD: whether it is about conveying visual information or whether broader interpretations should be incorporated into descriptions for audiences with visual impairments. Implications for practitioners: These findings indicate that further discussion is needed regarding the purpose of museum AD and, in particular, the way in which objectivity is contextualized. They raise questions about AD providing visual information and/or seeking to address a wider museum experience, including the stimulation of curiosity or emotion.

Introduction Time constraints limit the quantity and type of information conveyed in audio description (AD) for films, in particular the cinematic aspects. Inspired by introductory notes for theatre AD, this study developed audio introductions (AIs) for Slumdog Millionaire and Man on Wire. Each AI comprised 10 minutes of continuous description incorporating information about the film's visual style, fuller descriptions of characters and settings, a brief synopsis, and cast and production details. The AIs were tested with participants who are blind and have low vision. Method Twenty-four visually impaired volunteers listened to the AIs before or after watching the films with AD, and gave feedback about their experience, either at sessions organized at the University of Roehampton, United Kingdom, or at home. Results This was a small-scale, exploratory study that showed a positive response to the concept of AIs for film. Most participants felt the AIs helped bring the films to life and made them easier to follow. The majority of participants wanted AIs for other films. Discussion AD guidelines discourage describers from mentioning camera work, yet most participants reported that this information in the AI was not too technical, and that the proportion of the AI devoted to visual style was about right (14 of 20 for Slumdog Millionaire, 14 of 16 for Man on Wire). Only a minority felt that the AIs “told me things I could find out for myself.” This suggests that access to screen media for people with a visual impairment can be enhanced by additional cinematic and other visual information. Implications for practitioners Given the limited time available for description during the film itself, AD providers should consider the use of AIs as a complement to standard film AD. These audio introductions could be stand-alone or accessed from a website.

This article explores whether the so-called new “cinema of attractions”, with its supposed focus on visual effects to the detriment of storytelling, requires a specific approach to audio-description (AD). After some thoughts on film narrative in this type of cinema and the way in which it incorporates special effects, selected scenes with AD from two feature films, 2012 (directed by Emmerich) and Hero (directed by Zhang Yimou), are analysed. 2012 is a disaster movie aiming to thrill the audience with action. Hero is an equally visual movie but its imagery has an aesthetic purpose. The analysis investigates how space, time and action are treated in the films and the ADs, and how the information is presented in terms of focalization, timing and phrasing. The results suggest that effect-driven narratives require carefully timed and phrased ADs that devote much attention to the prosody of the AD script, its interaction with sounds and the use of metaphor.

Music has been an integral part of films from as far back as the silent movie era, where a piano accompaniment was used to build the narrative and an orchestra was used to drown out the sound of the projector. Music in films can encompass a wide array of elements, from a song to a whistle, and can play many roles, as has been commented upon by many researchers. Due to the relevance of music in film narrative, it is an element which merits attention when it comes to making a film accessible, be it through Audio description, Audio subtitling, Sign Language or Subtitling for the Deaf and Hard-of-Hearing. This article focuses on Audio description (AD) and how music is handled in audio described films. First of all, a brief review of existing AD guidelines will be presented in order to highlight the secondary role given to music insofar as these guidelines focus more on topics such as characters, locations, actions, credits, among others. This will be followed by an analysis of a corpus of audio described films, focusing on the treatment of songs with significant lyrics. Two different scenarios will be considered: on the one hand, films with song lyrics that can be understood by the audience because they share the same language and, on the other, English language films with song lyrics in a foreign language. Finally, this article proposes new approaches to the treatment of music in AD.

The filmic product undergoes a process of deconstruction and selection by the describer, who must decide which particular elements of the characters, costumes, set and action are to remain. Given its importance for professional practice, the acquisition of an appropriate yet flexible concept of relevance for audio descriptions is one of the main aims of any AD course, but how can translator trainees learn to decide what to describe? This article presents a teaching proposal aimed at tackling the problem of relevance in audio description through the combined use of evaluation criteria and film narrative.

Objectivity is a constant recommendation in audio description (AD) guidelines. However, some scholars have pointed to the advantages of a more subjective, creative, or narrative rendition. In this paper, we seek to determine to what extent subjectivity and objectivity coexist in Spanish filmic AD. In order to fulfil our aim, we operationalised objectivity by means of multimodal analysis based on Chaume’s (2004) classification of meaning codes. To operationalise objectivity through multimodality, we employed a qualitative content analysis and examined whether visual and acoustic information was rendered objectively or subjectively in a corpus of four Spanish AD scripts from Netflix. Our results show, firstly, that objectivity and subjectivity interact in the mobility, iconographic, and editing codes. Moreover, dissimilarities arise in the way these meaning codes are described in Spanish: movement is mainly described objectively, whereas iconography and editing are rendered either subjectively or objectively. In conclusion, we can state that neither is objectivity systematically applied in our AD scripts, nor is a purely narrative AD to be found, but rather a mixture of both coexists. Furthermore, this coexistence seems unequal, since different tendencies can be identified in the way that movement, editing, and iconography are described. Lay summary Audio description is an oral narration aimed at blind people that provides information about what is going on in an audiovisual product. The traditional view has been to inform objectively about the audio description contents, but some scholars support more subjective or creative alternatives. This papers seeks to observe whether audio description actually contains both subjective and objective elements. In order to find these elements, we used multimodality, a kind of analysis that shows how visual and acoustic information interact to create meaning. In this analysis we exposed instances of objective and subjective elements in a group of Spanish audio descriptions from Netflix. Our results show that there are objective and subjective elements when informing about movement, symbols, or spatio-temporal changes. However, these elements are not balanced, since movement is mainly described objectively, whereas symbols and spatio-temporal changes are described either subjectively or objectively. In conclusion, we can state that current audio description seems to be the result of a mixture of subjective and objective elements.

Abstrakt
 Cel badań. Przeprowadzone badania oscylują wokół zabarwienia emocjonalnego deskrypcji do reprodukcji malarskich. Punkt wyjścia dla rozważań nad audiodeskrypcją jest podział dokonany przez Romana Jakobsona, wedle którego audiodeskrypcja daje się sklasyfikować jako intersemiotyczny typ tłumaczenia audiowizualnego. W artykule zostaje podany zarys audiodeskrypcji w Polsce oraz jej ogólne ujęcie w dyskursie naukowym. Zwraca się uwagę również na dwa podejścia do tworzenia opisów audiowizualnych – niemieckie (zadaniowe i techniczne) oraz polskie (artystyczne i nastawione na wzbudzenie emocji u niewidomego odbiorcy). Celem badań jest ukazanie połączenia dydaktyki i przekazu emocji w audiodeskrypcji i określenie ich funkcji.
 Metodologia. Przedmiotem badań jest publikacja Beaty Jerzakowskiej Posłuchać obrazów, w skład której wchodzą deskrypcje do obrazów z Podstawy programowej dla uczniów gimnazjum i liceum. W ramach artykułu dokonana zostaje analiza jakościowa dwóch deskrypcji pod kątem językowym, para-tekstualnym oraz z uwzględnieniem elementów dydaktycznych.
 Wyniki. Audiodeskrypcja jest komunikatem emocji, która pomaga uczniom rozwijać swoją wrażliwość i poznawać nowe konteksty kulturowe.
 Wnioski. Deskrypcje spełniają funkcję artystyczną, ponieważ zastępują warstwę wizualną wyjściowego dzieła. Poza tym badane audiodeskrypcje niosą ze sobą funkcję edukacyjną, ponieważ stanowią uzupełnienie kształcenia literackiego i językowego uczniów. Kolejną funkcją audiodeskrypcji jest ich pragmatyzm – autorka książki świadomie łamie przyjęte standardy tworzenia audiodeskrypcji, dodając do opisów m. in. pytania retoryczne, metafory i synestezje. Ponadto dydaktyzm i próba przekazu emocji uwidaczniają się również w warstwie brzmieniowej tekstu oraz w muzyce wkomponowanej w każdą z AD.
 Słowa kluczowe: audiodeskrypcja, tłumaczenie audiowizualne, przekład intersemiotyczny, emocje.

1. INTRODUCTION term audio description refers to the spoken commentary is added to film soundtracks to enable visually impaired people to follow the action on the screen by supplementing the information is already available from character dialogue and other parts of the soundtrack. As this essay explains, there is currently a debate within this field regarding the possibility and desirability of objectivity in audio descriptions. Although this debate may appear to be of limited interest to anyone who is not directly concerned with the production and reception of audio descriptions, we feel it raises large and important issues are of relevance to all those who work in the much wider fields of narrative theory, stylistics and literary theory generally. We have in mind, in particular, our questioning of the apparently obvious distinction between descriptions of characters' actions and descriptions of characters' mental processes in fictional narratives. Specifically, this essay seeks to show how concepts from narratology and techniques from corpus linguistics can be applied to the analysis of audio description. We are concerned with the issue of what should be described in audio description for feature films, and how it should be worded. We introduce narratological concepts can help to better articulate this issue and to better analyze and compare examples of audio description. To complement the application of narratological concepts for a close reading of audio description samples, we show how corpus linguistics techniques can be used to learn about what is being described, and how, in a corpus of ninety-one audio description scripts. Our focus is on how an audio description utterance can, and, as it turns out, often does, describe a character's actions as depicted on-screen while simultaneously giving some information about the character's thoughts. Central to the practice and theory of audio description is the question of what should be included in a description (Vercauteren). Concerning the issue of what to describe, and how to describe it, some recent guidelines for audio description state: The best audio describers objectively recount the visual aspects of an image. Subjective or qualitative judgments or comment get in the way--they constitute an on the part of the describer and are unnecessary and unwanted ... Describers must differentiate between or (which requires an on the part of the observer) and the physical characteristics of or (which are more concrete and allow description users to conjure their own interpretations). (Snyder 17) While striking an exceptionally unequivocal tone, these guidelines reflect a point of view has been prevalent among audio description practitioners. Earlier guidance on standards for audio description provided by ITC is similar in advising not to interpret events or away the plot (ITC). ITC guidance also flags a contentious point about whether or not to give information that is not on the screen, without really explaining how to determine what is apparent. It seems to us such guidelines leave unanswered questions about what information is necessary for an audience to understand and enjoy a film, and what interpretation means in this context Also, and this becomes the main theme of the essay, with regards to what can be seen and hence described, we take issue with the supposedly clear-cut distinction is made between a character's emotion or reasoning (thoughts, which should not be mentioned according to the guidelines) and their apparent physical manifestations (actions, which may be described). This essay seeks to contribute to audio description theory and practice in two ways. First, we introduce more rigorously defined terms from narratology to clarify and simplify the issues alluded to in the previous paragraph. …

Audio description (AD) is an audiovisual translation (AVT) and media accessibility mode aimed primarily at visually impaired users. Traditionally, AD has tended to use images and some sounds as the source text to create the AD script (the target text), therefore AD is known as an intersemiotic translation. However, some researchers have focused on the theoretical feasibility of opting for the interlingual translation of ADs into another language. Moreover, research has explored the combination of AVT with language teaching and learning, a discipline called “didactic audiovisual translation” with branches like didactic audio description (DAD). In this proposal, we argue that DAD needs to consider both the linguistic nature and application of AD to better differentiate the didactic potential of intersemiotic and interlingual AD, since they have different applications for L1 and L2 learning and teaching. More specifically, intersemiotic AD seems to be suitable for learning L1, for instance, to consolidate grammatical structures, to proofread texts, to develop writing and oral skills, to learn about L1 linguistic variation, etc. On the other hand, interlingual AD can be used to learn about vocabulary and grammar in L2, to develop foreign language oral and written skills, to develop translation skills, etc. All in all, given the potential of DAD for language learning and teaching, this line of research should be further explored with other combinations.

Since 2007, there has been a systematic research conducted in the field of Audiovisual Translation (AVT) at ISCAP/ Porto Polytechnic Institute. At the time we embarked on a research endeavor focusing on audio description (AD), with the intent of systematizing AD guidelines, improving the AD process and reflecting on teaching/learning methods. This study presented at the Media For All conference, Antwerp, in 2009, aims to contribute to this project by focusing on issues linked to the generalized view that AD language should be ‘objective’ and therefore referential in nature, as stated in several guidelines. Indeed, the audio describer is even warned against expressing emotions or personal points of view. How seriously is this advice taken? Is this in fact what we should be teaching and doing? Are there identifiable common language functions in AD and are these clearly portrayed in the existing guidelines? In order to answer these questions, we assess the BBC Guidelines on the provision of television access services; the Audio Description International’s AD Guidelines Draft; the American Council of the Blind’s Audio Description Standards; the now extinct ITC Guidance on Standards for Audio Description; the Audio Description Coalition Guidelines for Audio Description, listing and contrasting their recommendations as to the ‘how’ and the ‘what’ of AD. Next, we compare these findings with randomly selected audio described feature films pertaining to the genres of drama, action, and suspense, namely Blindness, Revolutionary Road, The Happening, Body of Lies, The Eye, and Hancock. After analyzing both AD segments and movie clips in terms of visual rhetoric and Jakobson’s language functions, we propose that described movies stretch the concept of intersemiotic translation.

O Brasil apresenta um contingente de mais de 6,5 milhões de pessoas com deficiências visuais, sendo 582 mil cegas e 6 milhões com baixa visão. Neste universo, poucas iniciativas educacionais em geociências são identificadas, e, em geral, não fazem uso de vídeos educativos. O presente estudo tem como objetivo a apropriação do recurso da audiodescrição em vídeos educativos, com foco no ensino de ciências, especificamente, conteúdos relativos às geociências e meio ambiente. A produção de materiais de divulgação científica para deficientes visuais, com foco na educação básica, carece de recursos didáticos, tal como a audiodescrição, que auxiliem ao professor e propiciem ao aluno com deficiência visual acesso a vídeos encontrados no YouTube. A audiodescrição (AD) baseia-se em técnicas de transformação do visual em verbal, abrindo-se um leque de possibilidades de acesso à cultura e à informação, promovendo a inclusão social, cultural e escolar de indivíduos com deficiências visuais, e pode ser aplicada em espaços formais e não formais de educação. A audiodescrição, por se tratar de uma temática muito recente no campo social e científico, e que envolve a educação inclusiva, necessita de pesquisas sistemáticas, para que se possa desenvolver normas de audiodescrição que uniformizem a técnica em diferentes países em que a AD é aplicada. Assim, as atividades que permearam esse estudo foram efetivadas no âmbito do projeto “Cinco Pedrinhas”, componente do Programa Espaço Ciência Cultura Educação (ECCE), da Escola de Artes, Ciências Humanidades (EACH-USP). A audiodescrição foi desenvolvida para ser incorpora ao vídeo em stop motion publicado no YouTube, “Cinco pedrinhas saem em aventura”, propiciando assim uma narração descritiva em áudio. Para compor essa audiodescrição foram utilizadas referências nacionais e internacionais de técnicas que permitiram o desenvolvimento dos textos, além do acompanhamento de um audiodescritor para correção e direcionamento do trabalho, e estudos de campo por meio do acompanhamento de peças teatrais e musicais nas quais a audiodescrição estava disponível.

With the rise of streaming platforms like Netflix, viewers now have greater access to language and accessibility features. Audio description (AD) translates visual information into spoken language, enabling access for viewers with visual impairments. To match the experience of sighted audiences, AD must convey all visual cues, including physical appearances. Despite its importance, this area – where accessibility and inclusivity intersect – remains understudied. This paper investigates how Netflix’s AD represents different body types, physiques, and visible disabilities among able-bodied and disabled individuals. Using a qualitative descriptive approach, the study analyses AD in two series, one film, and one documentary to assess clarity, accuracy, consistency, and implications of these descriptions. Findings reveal inconsistencies in describing body shape and size, with such references being rare and often contingent on plot relevance. Descriptions typically appear in contexts tied to body image or physical performance, especially relating to athleticism and (dis)ability. In cases involving disability, AD often describes mobility aids rather than the individuals themselves – an approach that avoids potentially offensive terms but risks reinforcing the taboo around discussing disability. The study suggests that these inconsistencies stem from both linguistic limitations and a reliance on perceived relevance. It concludes that prevailing discourse and stigma around describing human bodies complicate AD practices. Further research is needed to refine AD guidelines, ensuring they reflect the perspectives of both those being described and those who rely on AD.

In the past few decades, studies on how verbal metaphors shape our thoughts have been widely discussed in the filed of cognitive science, cognitive linguistics, psychology, mathematics, art, and philosophy, etc. Under this view, “non-verbal” metaphors have also received considerable attention, while previous studies mainly focused on the mental processing and one-on-one correspondence between language and other mode, especially the pictorial mode. However, this study believes the relations between multimodalities can be viewed as a complex continuum of various kinds of combinations. Therefore, this study proposes an alternative interactive interpretation model, the Multimodal Fusion Model, to account for the one-on-many relationship between language and multimodality (visual and aural modes) and to explore how certain topic/issue has been represented in various forms in distinct genre. Based on the evidence from the multimodal genre of political cartoons and poetry music, we examined (1) how the hotly debated topic, the U.S. beef import issue, has been represented in various political cartoons in different newspapers and (2) how a classic poem (i.e. Serendipity) has been conceptualized in four versions of songs. We further incorporate the audience response analysis from the interview surveys to examine how the audience actually interprets the multimodal genre. Our findings also show how prominent multimodal features and multimodal cues (visual/aural/verbal cues) help facilitate the interpretation process of multimodal metaphor in specific genre. This study constructs a multimodal corpus involving 56 political cartoons with regard to U.S. beef import issues as reported in two dominant Taiwanese newspapers, the Liberty Times and United Daily News. Our corpus shows that multimodal fusion combining the visual mode, verbal mode, and the conceptual level evolves from two metonymic-metaphoric networks, i.e., related metonymic network and diversified metaphoric network. The result demonstrates that multimodal fusion is a significant and recurrent representation technique in the genre of political cartoon and has the cognitive function of encapsulating the abstract complex political debates efficiently with irony and humorous effect. In addition, this study emphasizes the important role of metonymy and demonstrates how metonymies and metaphors are interwoven in the process of multimodal fusion, which underlies the metaphorical mappings of conceptual scenarios related to “POLITICS IS GAME” and “POLITICS IS WAR”. Although the critical messages and distinct political stances in two newspapers both emerged through multimodal fusion, they are highlighted and contrasted through prominent visual features and verbal context in political cartoons. Based on the evidence from four versions of poetry music composed for the poem “Serendipity” by Hsu Chih-Mo, this study provides a metaphor-based musical analysis under the scope of multimodal fusion model. We discovered the “chain of metaphors” as an interactive and echoing relationship between metaphors that creates a holistic representation of multimodal metaphor combining the verbal and aural mode in the genre of poetry music. Furthermore, the results show the metonymic representations of metaphor in poetry music have been realized through the “pictorial effect” of partial visualization of cloud image on the score representing CLOUD metaphor and aural simulation of wave rhythm representing SEA metaphor, etc. Through cross-comparison analysis, we propose two major aspects of metaphor, the “concrete image” and “abstract metaphorical extension”, have been highlighted differently through musical techniques and features such as dynamics, pitch, and timbre, etc. in four versions of songs. Moreover, this study provides a concrete-abstract continuum as a categorization of composing patterns and musical effects. Viewing from the embodied perspective, our findings also show that the concrete relationship of space and abstract attributes of emotion can be mapped onto the representations of prominent musical features in poetry music. This study further incorporates the quantitative and qualitative analysis from audience response survey. The results show that higher level of audience interpretation in political cartoons relies on the effective visual recognition of caricatures, comprehension of verbal and contextual information, and interactive connection of the verbal mode, visual mode, and conceptual level. On the other hand, the higher level of audience interpretation in poetry music is related to the distinction between the concrete image and abstract metaphorical extension. The results also suggest that the audience’s preference is relevant to the correspondence between the verbal and aural mode, which implicitly reflects how the audience “feels” about the musical effect and aesthetic functions created from multimodal fusion in different versions of songs. From the perspectives of researcher and audience, this study investigates how verbal mode, visual mode, aural mode, and conceptual level are interacted, integrated, and interpreted through multimodal fusion. We hope to shed light on the interdisciplinary studies on multimodality and the interface between the cognitive mechanism and pragmatic multimodal use of language, picture, music, and the aesthetic experience of the multimodality appreciation process.