Intersemiotic Translation Research Articles

Research on English Translation and Overseas Dissemination of Chinese Calligraphy

This article explores the aesthetic value and profound connotations of Chinese calligraphy as well as the translating strategies for its overseas dissemination. By comparison and contrast of different translation such as literal translation, free translation and intersemiotic translation, the article summarizes that the theory of intersemiotic translation provides a new perspective and methodological guidance for translating Chinese calligraphy into English. The article also analyzes the current situation and challenges during disseminating Chinese calligraphy overseas, and proposes combining new media and network technology is an effective means for overseas dissemination of Chinese calligraphy.

Translation of English Instructive Images of COVID-19 in Social Media into Arabic: A Semiotic Study

This study investigates the intersemiotic translation process of specific images used to instruct people concerning COVID-19 disease that are posted on different social media. It also attempts to identify translation strategies employed by translators to convey the meaning from one non-verbal form into an Arabic verbal form. This study depends on a multimodal perspective that combines comprehending the connotative meaning of the image and analyzing it into a different verbal expressive form. It is hypothesized that translators are aware of the translation strategy through which the expression of an image is communicated successfully into Arabic verbal expressions. Translators employ Stetting's strategies that are restricted to media translation and journalistic translation, which is used for transferring the meaning of different forms of media. Five translators participated in this study to translate eight images into Arabic. This study concludes that a sign's detonative and connotative meanings are rendered, whether positively or negatively, to transfer the meaning, whether directly or indirectly.

Crossing boundaries in an age of post-translation studies

Abstract The cycle of symphonic songs Das Lied von der Erde was based on Die Chinesische Flöte, a German version of classical Chinese poems. As a result, Die Chinesische Flöte crossed the boundary between literature and music, encompassing intralingual, interlingual, and intersemiotic translations. While comparative textual analysis reveals that these two works made changes to the original Chinese texts, intersemiotic translation illustrates that the borders between verbal texts and nonverbal works are fluid. This post-translation study traces the translation context of Das Lied, maps Die Chinesische Flöte’s transitions between literary and musical polysystems, and traces Das Lied’s afterlife in the creation of new artworks. It concludes that poetry, music, and other art forms can mutually translate, complement, and become afterlives of one another.

Unveiling hybrid culinary cultures in Macao: a corpus-assisted multimodal approach to the translation of Macanese cuisine

ABSTRACT This study investigates the special foodscape and hybrid culinary cultures in Macao through the translation of Macanese cuisine. It employs a corpus-assisted multimodal approach, with the concept of intersemiotic translation and multimodal theories as the theoretical basis for analysis and discussion. The data under investigation is a self-built multimodal database of Macanese cuisine translations. The study first examines the ingredients and cooking methods in Macanese cuisine dish names and then delves into the translation strategies and methods used in the translations across Portuguese, English and Chinese. Multimodal analysis is also conducted to explore the intersemiotic translation between the verbal and visual materials of Macanese cuisine. The research results show that hybrid culinary features are represented in the ingredients, cooking methods and translations of Macanese cuisine, constituting an inclusive, integrated and diversified foodscape in Macao. Moreover, the intersemiotic translation between the verbal and the visual could highlight particular ingredients and cooking methods, showcase the enticing appearance of the dishes, and convey the hybrid culinary cultures in Macanese cuisine. This study holds the potential to enrich research on food translation from a multimodal perspective, while also contributing to the translation and cultural transmission of Macanese cuisine and culinary cultures in Macao.

Birds of Feathers may not Flock Together: Avian Imageries in Contemporary Arab Diasporic Novels

he purpose of this paper is to examine how birds and avian metaphors are used by Arab writers in diaspora to reflect themes of exile, displacement and dispersion. In particular, in Inaam Kachachi’s The Dispersal (2023) and Walid Nabhan’s Exodus of the Storks (2021) birds and avian images are thematically and aesthetically significant motifs since the covers of some editions of the two novels feature a scene from each novel in which birds are of great importance to some characters, key events and sociopolitical, historical and cultural contexts of the texts. The deployment of these imageries on the covers of some editions represents a relatively recent tendency adopted by Western publishers to reflect the complex nature of literary representations of recent developments in the Middle East rather than relying on a long history of Orientalism to mediate this process of presentation. Furthermore, the study draws on the tripartite division of the notion of translation by Roman Jakobson (1959), particularly, his notion of the intersemiotic translation to highlight the links between a novel’s cover design and the themes that it depicts.

Intersemiotic Translation: Mat Dollah’s Batik Paintings of the Malay Folktale “Puteri Gunung Ledang”

In translation studies, intersemiotic translation, which is the transference of a verbal sign system to a non-verbal sign, vice-versa or between two non-verbal modes, is a dynamically creative form of translation often involving literary works. Intersemiotic translations make it possible for written source texts to be brought to life in various artistic ways; they can usurp the power of the written word to transfer sense and meaning through unforgettable images from one culture to another. Folktales, documented in writings, have been translated into many different forms, such as in performing arts, animations, paintings, and illustrated books. This article examines the intersemiotic translation of the evergreen Malay folktale, “Puteri Gunung Ledang” or “The Princess of Mount Ledang” recreated in the form of batik paintings by the batik artist, Mat Dollah. The first specific objective of this study is to identify the variant and invariant features in Mat Dollah’s representation of the folktale by comparing his batik paintings to its original written account. The second objective is to investigate Mat Dollah’s skopos or purpose for recreating the folktale in his batik paintings in the ways that he has represented it. Six selected batik paintings of the folktale are analysed using Kress and Van Leeuwen’s (2006) multimodality approaches in visual grammar focusing on social actor representations and viewer networks (2008), colour (2011) and modality to fulfil the first objective. Transcripts from an email interview with the painter are analysed to fulfil the second objective. In this study, Vermeer's Skopos Theory underpins the overall discussion. The findings show that one invariant core links the batik paintings to the written source text. Next, the skopos or purpose of the artist in producing these recreations in his own unique ways is strongly influenced by his personal experience, his work as an artist, and his perspectives of the world. Two key insights can be surmised about intersemiotic translations from these findings. Firstly, it is the translators’ intent that determines the translation strategies used, ultimately shaping the final translated product. Secondly, this makes the translator’s presence highly visible in intersemiotic translations.

‘There are no restrictions on me’: a netnographic approach to amateur subtitling from the perspective of translanguaging

ABSTRACT Amateur audiovisual translation is experiencing an increase in both prominence and scholarly attention, rejuvenating studies into audiovisual translation by shifting focus to earlier overlooked aspects of intralinguistic, interlinguistic, and intersemiotic translation activities, often sidelined in favour of monolingualism. Drawing on the netnographic data from the self-subtitling vlogger @成员外 on the popular Chinese video-sharing platform Bilibili.com, this study examines how and why the vlogger utilises multiple communicative resources to perform subtitling in amateur online settings through the lens of translanguaging. The results reveal that the translator employs source languages, local vernaculars, internet buzzwords, trans-registers, trans-modalities, and trans-discourses in his subtitling activities. These manipulative boundary-transcending transcreations not only enhance accuracy and expressiveness but also facilitate cultural mediation, and ultimately, foster the co-construction of an affinity space configured by a spirit of participation, collaboration, and mutual understanding. This study, as an attempt at the interdisciplinary study of applied linguistics and translation studies, sheds light on the production of audiovisual content in the media localisation industry and provides a solid foundation for future inquiries and explorations in this burgeoning field.

Double strategies for reproducing multimodal puns in interlinguistic translation: Experimental research

The research is dedicated to exposing the specifics of translating multimodal puns based on the retrospective experiment. A multimodal pun is defined as a creolized/ polycoded formation substantiated by a specific type of ambiguity consisting of two inhomogeneous semiotic modes. The material for our research was provided by verbal-visual puns functioning as separate texts. Each multimodal pun results from intersemiotic translation when the signs of one semiotic system are transformed into the signs of another. For the vast majority of multimodal puns, verbal signs are interpreted into pictures, but the opposite cannot be excluded either. The role of the visual component is twofold. It can be creative when the picture is part of the ambiguity mechanism, or it can be amplifying when the picture accentuates the verbal wordplay, not participating directly in the creation of ambiguity. Hence, our first hypothesis is that multimodal puns with an amplifying visual component are a lesser challenge for translators than those with a creative one due to the absence of the necessity to coordinate verbal and visual modes in the target text. The research aims to identify the strategies of interlinguistic translation of multimodal puns and the factors that determine them, particularly the impact of the visual mode on the translator’s decision-making. Hence, our second hypothesis is that multimodal puns require double strategies that would allow to correlate the reproduction of the elements of two different semiotic systems. The analysis of the subjects’ translations, as well as their reports received in the course of the delayed retrospective experiment, confirmed both hypotheses. Disclosure Statement The authors reported no potential conflicts of interest. * Corresponding author: Oleksandr Rebrii, 0000-0002-4912-7489o.v.rebrii@karazin.ua

Blurring borders in intralingual translation research: the case of journalistic translation

ABSTRACT Intralingual translation research is on the rise, as evidenced by the numerous recent publications and academic events on this subject. Some scholars have (re)defined and typologized intralingual translation, attempting to shed light on an otherwise seemingly nebulous concept. However, in view of the complexity and diversity of the practices that can be studied under the umbrella term of intralingual translation, a blurred borders approach might be preferable. In the present paper, this position is supported by means of two case studies on (intralingual and intersemiotic) journalistic translations or remediations for social media. The two objects of study are an infographic by Mexican information design company Pictoline and a video by British science magazine New Scientist. A comparative analysis of these multimodal and multimedia texts and their news sources illustrates the fuzzy borders between intralingual and intersemiotic translation, as well as between the different types of the former. If we wish to describe intralingual translation separately from the other Jakobsonian types, this can be done on the basis of its aim or purpose, its target users, its text type or field, or its use of different media or technologies.

What to translate and how to translate in audio description: a case study of the Oscar-winning animated film Feast

ABSTRACT Introduction: Audio description (AD) enables access to blind/low vision audiences by translating visual content into narrated descriptions. This study examines AD translation approaches for the Oscar-winning animated short Feast. Methods: Chinese and English AD scripts from Feast were analyzed to compare priorities in content selection and patterns in translation techniques. Results: Both versions prioritized graphics and dynamic symbols when deciding what to translate. Shared translation techniques included compensation, iconic description, and substitution. Differences emerged with more reduction, adaptation, and technical descriptions in the Chinese AD, versus more generalization in the English AD. Discussion: Disparities highlight the linguistic and cultural factors integral to effective AD translation. Implications: This analysis offers practical customization guidelines for AD providers and theoretical insights into intersemiotic translation complexity. Further comparative research could support tailored AD practices and accessibility for diverse global audiences.

Translating artworks

Abstract This contribution, which sits at the intersection of translation studies and museum studies (MS), seeks to explore multiple forms of interpretation developed by museums, whereby interpretation is considered, from an MS perspective, as a variety of museum aids creating a context and conveying mediated meanings about the objects on display. The focus is on individual interpretative texts produced by an Italian art gallery, the Pinacoteca di Brera, as the results of various interpretation processes about the same piece of art: the traditional label, the online description, the general audio guide, and the audio description for the blind and the visually impaired (the latter comprising both the visual description itself and the art historical description). A selection of texts in Italian and their translations into English describing four artworks from the museum collection are compared in a bid to shed light on the distinct layers of interpretation and ways of translating and representing the objects for different expected audiences through intralingual, interlingual, and intersemiotic translation practices.

On Subtitle Translation of American TV Series from the Perspective of Functional Equivalence Theory

With the development of globalization and cross-cultural communication, more and more artistic works are introduced into the promising market, China, including film and television works, music and novels. Of which, American TV series have become one of the hottest subjects, which arose enthusiastic pursuit and discussions among Chinese. Compared with domestic TV series, it contains more exciting plots and beautiful background music. What’s more, the quality subtitle is an essential part of this introduction. Regarding the theory put forward by Roman Jacobson, the founder of Prague school, translation is divided into three categories: intralingual translation, interlingual translation and intersemiotic translation, and subtitle translation is kind of interlingual translation. It means an interpretation of source language by means of target language. For film and television works, subtitle acts as a bridge between source language and audiences of target language as well as playing the role of communication and transformation of original emotions and image, which is the best illustration of functional equivalence theory developed by Eugene Nida. Differentiate from traditional literary translation, subtitle translation has its own unique features despite its being an emerging field. It has spatial and time constraint and must reproduce the content of source language as accurately and elegantly as possible within the limited time, as a result, this paper intends to carry out a further study on subtitle translation and functional equivalence theory. In 2018, the American TV series Killing Eve created a fad at home and abroad and the successful subtitle translation definitely contribute to it. In my perspective, its subtitle translation can be considered the great illustration of the principles and requirements of functional equivalence theory, which is a best case to study. From the perspective of functional equivalence theory, the paper has potential to produce a better understanding of subtitle translation's classification, features and related study at home and abroad. Give priority to the role of functional equivalence theory played in subtitle translation so as to give a boost to its development.

Opening the boundaries of a literary text: Anton Chekhov’s novel “Kashtanka” and its film adaptation in teaching Russian as a foreign language

The article describes the work with a literary text and its adaptation in the classroom of Russian as a foreign language. In particular, we focus on the ways of understanding a literary text by including a film adaptation into classroom activities. Special attention is paid to the characteristics of the intersemiotic translation and its basic concepts: adaptation, film adaptation and interpretation. For the methodology of teaching Russian as a foreign language, this distinction is conditional and is of great interest from the point of view of understanding the idea and content of the literary text and the peculiarities of its interpretation. When working with international students, the use of screen versions of Russian literature texts helps, on the one hand, to simplify the process of understanding their meaning and key points by visualizing verbal and nonverbal components of the text, on the other hand, to complicate this work - to increase the amount of information provided. The present paper presents the algorithm of working on the text of the short story “Kashtanka” by Anton Chekhov and its film adaptations and formulates sample tasks for classroom and independent work.

Mediated Translation between Italian and Russian Languages

La forza del destino was commissioned to Giuseppe Verdi by the St Petersburg Imperial Theatres in 1861, and premiered in November 1862, at the Bolshoy Kamenny Theatre. The opera was written on a libretto by Francesco Maria Piave, and draws its subject from the Spanish drama Don Álvaro o la fuerza del sino, by Ángel de Saavedra, through an Italian version by Faustino Sanseverino, and with insertions of fragments from Schiller’s trilogy Wallengstein. In this paper I analyse the genesis of the opera, considering the typical derivation of Italian operatic librettos from pre-existing literary works, a process that combines mediated and inter-semiotic translation. This allows me to show the artistic resources offered by mediated translation in the field of stage literature. On another level of discussion, given that for the Premiere Apollon Grigor’yev translated the libretto from Italian to Russian, I consider the role of translation on the reception of this subject by the local audience.

Book 1 and season 1 from the series “Bridgerton” intersemiotic translation and ethnic-racial issues

This article analyzes the first season of the Netflix series "Bridgerton," which was adapted from the first book in the "Bridgerton" series, "The Duke and I," written by Julia Quinn. It does so by drawing on a variety of viewpoints to offer fresh perspectives and promote intercultural conversations. The categories of "alteration," "omission," and "addition", adapted from interlingual translation, are used to discuss the changes that resulted from the translation of one sign into another. In this work of bibliographical origin, writers with experience in translation were chosen, and as the study includes a case study, books, interviews, and articles from online magazines served as sources, giving a more in-depth understanding of the object mentioned. The adaptation from a literary work to an audiovisual one, as suggested in this article, results in the source work being transformed, through alterations, additions and omissions. The additions and changes to the series opened a discussion about racial issues and, since this work involves qualitative research, it points out how these modifications affect both the media and society, highlighting the significance of having conversations about the matters above mentioned. Additionally, the racial representativeness shown in the series had wide acceptance and support from the general audience. All in all, this article develops topics such as: intersemiotic translation, ethnic-racial issues and analysis of the differences exposed in the series adapted from the book “The duke and I”.

Semiotics of Machinic Co-Enunciation

In this paper, we propose a semiotic study on generative artificial intelligences, considering the work of Midjourney and DALL·E, that are computational devices capable of producing original images on the basis of the training they have received on large databases of visual, verbal and multimodal documents. The aim is to frame their functioning from a semiotic point of view and to describe the operations that can be performed during image composition, through the options available on the two platforms. To achieve this objective, the paper will be developed into three main parts. In the first part, we will provide a general contextualisation of the relationship between semiotics and artificial intelligence, in the broad sense. Starting from Pierluigi Basso Fossali’s (2017) assumptions about a semiotic perspective understood as the study of the social organization of meaning, AIs reconfigure the thresholds between the four dimensions he defined (perception, enunciation, communication and transmission). By considering the transmission and perceptual dimension, we will define the phases of the database construction and AI model training as pertaining to an archival (distributed) perception. In the second part, we will deal with the dimension of enunciation. On the one hand, we will describe the functioning of the diffusion models guided by human prompts. On the other hand, we will show the limits and potential of these AIs, through a discussion of the operable commands and the results obtained, in light of the experiments we have carried out over the past months (August 2023-May 2024). In particular, we will test how generative AIs produce images on the basis of prompts containing the styles of specific artists, how they fuse different styles together, and how they work on visual stereotypes. In the third part of the paper, we will focus on the relationship between verbal description and visual generation in order to provide, in accordance with the perspective of the intersemiotic translation, a new research object and a new methodology.

Intersemiotic translation of Yixiang (Image): A case study of Tang Poetry in paintings

This paper focuses on intersemiotic translation of Yixiang, or image in western literary term, from poetry to painting by taking Tang Poetry in Paintings as a case to exemplify the translation strategies of Yixiang from verbal sign system to visual sign system. Furthermore, the meditative role of the visualized images in the target context will also be touched upon. A qualitative approach is applied in the study. Typical examples will be chosen for case analysis. It is found that the strategies of equivalence, focalization, integration, montage, addition, contrast and omission are employed by the painters. The effects of those visualized images varied in different cases. Overall, cooperating with the interlingual translations, visual languages such as color, shadow, body language of figures, picture structure are employed to help English readers better receive the poems.

From biblical texts to Balinese paintings

The intersemiotic translation is defined as the interpretation of linguistics signs by means of non-verbal resources. This study aims to investigate the intersemiotic translation of linguistic texts” The Parable of the Ten Virgins” into an image system of Balinese Paintings made by Christian artist Nyoman Darsane. The approach applied in this study is derived from multimodal social semiotic theory in accordance to Halliday’s Systemic Functional Theory (SFT). The source of data is biblical texts which were taken from the New International Version of the Parable of the Ten Virgins of Matthew 25 verse 1 until 13 and its intersemiotic translation in the form Balinese painting. The result of analysis has demonstrated that the metafunctions principle of the different semiotic systems which in this case is texts and painting have complementary and compounding meaning to each other in all levels of meaning including experiential, interpersonal, and textual meanings. Even though there are some untranslated keys of information in those levels of meaning as there are omission strategy implemented from source text to the target image for various reasons such as presenting only essential information.

Intralingual and intersemiotic translation for accessibility in educational and social environments - focus on legal language

Over the years, what is normally referred to as visual law has increasingly developed and some scholars have been tentatively experimenting with legally binding comics, although in very limited numbers. This article posits itself as part of that tradition, while taking the issue a step forward, further developing the academic discussion around the notion of accessibility in legal language and emphasising the importance that different forms of translation can assume in this context. The article thus discusses the impact that intralingual and intersemiotic translation can have in both “educational” and more “social” contexts, finally providing two brief examples of how these forms of translation can render legal discourse more accessible and understandable to a great variety of receivers, regardless of their age, level of education, cultural origin, etc.

Translation solutions used in the binary and intersemiotic translation of the graphic novel about the Marvel universe

The article discusses the key translation solutions that have been applied by translators in the binary and intersemiotic translation (film adaptation) of a graphic novel about the Marvel universe. The object of the study includes occasionalisms, substandard lexis and onomatopoeia in the graphic novel “X-Men: Battle of the Atom” as one of the most popular within the analyzed universe. Within the framework of the research, such linguistic methods as the continuous selection method, the contextual analysis method, the structural and semantic analysis method, the descriptive method, the comparative analysis method, the contrastive analysis method, as well as quantitative calculation methods are used. The authors conduct a comparative translation analysis of the translation and the original of the comics “X-Men: Battle of the Atom”, during which they identify occasionalisms and distribute them into certain thematic groups. The study highlights the main types of onomatopoeia typical of the English-language graphic novels. The calculation and analysis of translation solutions used in the transfer of onomatopes used in the comics into the Russian language are carried out. Attention is paid to the main types of stylistically substandard lexis – vulgarisms and swear words, which are also considered in terms of translation from English into Russian. When analyzing the transformations of the verbal component of a graphic novel in its intersemiotic translation (film adaptation), the authors focus on the occasional lexis, since in this type of translation only occasionalisms can have correspondences. The substandard lexis in the film adaptation is omitted, and when translating onomatopoeia, the verbal component is converted into the sound component of the film text. In the intersemiotic translation, only a part of the occasionalisms is preserved, which is conditioned by the reduction in the number of characters and significant locations.