Visual Translation Research Articles

Talking Face Generation With Audio-Deduced Emotional Landmarks.

The goal of talking face generation is to synthesize a sequence of face images of the specified identity, ensuring the mouth movements are synchronized with the given audio. Recently, image-based talking face generation has emerged as a popular approach. It could generate talking face images synchronized with the audio merely depending on a facial image of arbitrary identity and an audio clip. Despite the accessible input, it forgoes the exploitation of the audio emotion, inducing the generated faces to suffer from emotion unsynchronization, mouth inaccuracy, and image quality deficiency. In this article, we build a bistage audio emotion-aware talking face generation (AMIGO) framework, to generate high-quality talking face videos with cross-modally synced emotion. Specifically, we propose a sequence-to-sequence (seq2seq) cross-modal emotional landmark generation network to generate vivid landmarks, whose lip and emotion are both synchronized with input audio. Meantime, we utilize a coordinated visual emotion representation to improve the extraction of the audio one. In stage two, a feature-adaptive visual translation network is designed to translate the synthesized landmarks into facial images. Concretely, we proposed a feature-adaptive transformation module to fuse the high-level representations of landmarks and images, resulting in significant improvement in image quality. We perform extensive experiments on the multi-view emotional audio-visual dataset (MEAD) and crowd-sourced emotional multimodal actors dataset (CREMA-D) benchmark datasets, demonstrating that our model outperforms state-of-the-art benchmarks.

GraphicsAssist: Visual Artist’s Assistant for Narration-to-Illustrative Graphic Composition

Storytelling is an art that very few can carry with ease, putting in proper context the feelings and emotions of the characters, their setting, interactions, etc. This work attempts to recreate storytelling to form a graphic illustration for users so that the essence of the story is not lost in transference from one person to another, from storyteller to graphic illustration reader/user. ‘GraphicsAssist’ is an artist assistance system for graphic artists to help them generate certain options for usage as graphic frames. Text analysis has been exploited to understand contextual requirements or storytelling. Natural language-based learning has been applied to this requirement. The transfer-learning-based model is used to find suitable matches for the characters and visual environment settings. Illustrative drawings and their tags are considered a part of the frame database. Once the narration entities of the story are extracted, they are matched with the drawing tags and presented to the designer for framing and then panelling of graphic illustration. The chosen frames are displayed in panels in grid format. In its current form, manipulation of the selected drawings for the task of forming frames is, as yet, restricted. The aesthetic transition from frame to frame is yet lacking and may be integrated into later works. The work of this artist’s assistance system is validated by the test group, which gave feedback on some pre-defined parameters. Though the system has its drawbacks, it is the first step towards such a system that provides a computational translation model for translating a textual narration to a set of visual composition frames. This work aims to capture the nuances, characters, emotions, story setting, etc., from textual narration as an input of the story and convert it to the graphic illustration in 2D form by incorporating illustrative drawings as its visual translation.

Courtiers in Translation: Rubens’ Portrait of Baldassare Castiglione after Raphael

Around 1519, Raphael painted a portrait of his friend Baldassare Castiglione, author of the influential Book of the Courtier (1528). This portrait was later reproduced by Rubens, who may have seen the painting by Raphael, or a copy of it, during his Italian sojourn. Tracing the tensions between Rubens’ “copy,” Raphael’s portrait, and Castiglione’s book, this essay explores how Rubens’ painting problematizes the notion of visual translation in early modern Europe. I argue that his version is not merely a portrait of one person, but a conflation of four presences: the ideal courtly figure; the writer-courtier Castiglione; and the two court artists Raphael and Rubens. Ultimately, the painting represents a paradigm of collaboration and rivalry that underpins the process of translation, one involving multiple hands, works, and techniques, and thus defying the label “copy.”

Film + Design

The design of scenography, staging, lighting and objects for representation and presentation within the film genre forms one of the most seductive areas in the practice of interior design.?The merging of space and movement within this medium provide context and site for the visual translation of narrative, engagement of image and spectacle and discourse surrounding the iconography and imagery of history, culture and society. This studio, delivered to second and third year interior design students at the University of South Australia, delved into the multi-disciplinary practice of the film process. It introduced students to the various roles and responsibilities of the designer working within the medium of film. It attempted to meld the theoretical and practical through a series of design projects which analysed and developed design proposals while challenging traditional realms of interior design practice. Four specific theory lectures were developed to provide a theoretical reference for the project work. These were structured to explore notions of montage, place, and fragmentation through representation and imagery. To augment these lectures and translate existing skills into the conventions of film design practice, practical workshops exploring the techniques of script breakdown, drafting, lighting and packaging formed an integral part of the lecture series. To develop critical analysis and observation of existing practices, students were asked to watch a minimum of ten films - either from the supplied listing' or as proposed. They were asked to comment both in visual and text form, on the various devices used to visualize the narrative in each film. To do so, they identified the broad themes of the film and documented design methods used to represent these themes - form, lighting, spatial sequencing, etc Gaining an understanding of a body of film works allowed students to engage in both theoretical and practical approaches to the design process in a reflective and responsive way.

Blasted, Bound, and Bald: the Abject Properties of Witch Hair

The witch’s blasted hair in Early Modern visual culture is moving and evolving. Her hair is untamable, animated by a wind and a force unknowable to the viewer. The artists who captured this “blasted” hair in the 16th century, including Hans Baldung Grien, Hans Sebald Beham, and Albrecht Dürer, found themselves working in opposition to what appears, at first glance, a formidable “other:” women’s agency and sexuality, frozen in bodily form as strands of hair. Little research has been dedicated to an earnest analysis of the witch’s amorphous hair so this paper consists primarily of original research and the application of philosophical and anthropological theory. Julia Kristeva’s theory of abjection is a key tenet of this paper’s central argument. The witch is not an “other” directly opposed to a mainstream, she is a third intermediary thing: Kristeva’s “abject.” She is a personification of the skin on the surface of sour milk, instilling disgust in the potential consumer. She is the abject precisely because she is embedded in the object; her hair is the most explicit vehicle for this argument. Friedrich Speevon Langenfield’s accounts of forced hair removal and contemporary literature from Raymond Firth, Barbara Baert, and E.R. Leach inform this reading of witch hair as the abject. This paper concludes that the visual translation of blasted hair is an attempt to remove it and its female sexual connotation from the imagination to a more accessible visual medium. The artist’s renderings are meant to turn the abject into an easily digestible object, a complete “other.”

Research on the Integration and Innovative Expression Methods of Visual Elements of Popular Music in Multimedia Era

Abstract Visual art and music art are two typical media, and the progress of multimedia technology has prompted the cross-border integration between them. This paper tries to visualize the visual translation of music acoustic information and emotional experiences through audiovisual mapping design. The extraction and analysis of musical and visual elements in the music visual mapping model are summarized and analyzed. Combined with the visual expression method, a mapping pattern framework that can be reused in music visualization design is proposed, and two types of mapping patterns are classified on top of it. Based on the four basic attributes of pitch, chord, tone, and timbre of popular music, corresponding visual elements are designed. Finally, the impact of the two mapping modes on the functional and aesthetic effects of music visualization is experimentally demonstrated and evaluated, and it is found that the mean value of the scaling index of the original HCR sequence of the pop music clip is close to 0.864, while the mean value of the disturbed sequence is 0.504, which is a referable evaluation result for determining the appropriate mapping type when designing music visualization.

Visual Translation of Auspicious Beliefs in Quanzhou Xi Culture from the Perspective of Man-Machine Collaboration

Visual Translation of Auspicious Beliefs in Quanzhou Xi Culture from the Perspective of Man-Machine Collaboration

Memory studies and design? The mnemotopic approach

At the beginning of the fourth wave of memory studies, the relationship with design appears poorly explored, although the memory project is fundamental for the contemporary and interdisciplinary past reframing. From this perspective, the article proposes an intrinsic but not yet evident link between memory studies and communication design for the territory, a young discipline that has taken up the “stratified place” as its own specific dimension, focusing on its valorization and visual translation. This novel connection is highlighted by the mnemotopic approach in which place and memory coexist in a unique meaning formation, supporting the remembrance of the territorial experience. Through a lexical transfer of the term mnemotope, generally used in other fields of knowledge (e.g. cultural anthropology), to design, it becomes a performative concept integrated into processes and artifacts as an active part of project development. The mnemotope as a plural object of territorial interpretation, sailing from Nora’s lieux de memoire, passing through Assmann’s cultural memory reflections, and landing in communication design, can be adopted as an alternative interpretative criterion that not only proposes a resematization of the memory of places but can be considered as a real medium for exploring the past that can also operate on a didactic level by being included in experimental design courses. In this context, the article will show how the mnemotopic approach has been developed during my personal doctoral journey and pedagogically implemented by the DCxT research group of the Design Department of Politecnico di Milano.

Visual Translation: Illuminated Manuscripts and the First French Humanists. By Anne D. Hedeman

<i>Visual Translation: Illuminated Manuscripts and the First French Humanists</i>. By Anne D. Hedeman

Construction of a platform for displaying dialectalized graphic design based on virtual reality technology

Abstract This paper uses a semantic analysis of dialects, and a dialect word vector model is constructed. Based on the dialect word vector, the visual translation graphics of the language are used as the basic element to realize dialectalized graphic design. Secondly, the feature point matching is combined with the SIFT algorithm, where the key point is to elaborate the projection transformation matrix by the polar geometry and solve the projection transformation matrix. Then, the Jacobi determinant determines the image distortion to determine the contrast correction of the two graphics. A dialectalized graphical display platform uses virtual reality technology to achieve human-computer interaction. After testing the platform, it was obtained that the error rate generated by the display method was between 0.9 and 1.3%, the display accuracy was about 95.8%, the running time required for the display method was between 2.8s and 3.9s, and the platform display performance was good.

Sacred King in the Shah Tahmasp Shahnama

This paper aims to trace the pre-Islamic Zoroastrian concept of “royal divine glory” (farr) through its visual translations within the Iranian manuscript cultures in the Safavid period, specifically in the illustrations of the Shahnama-yi Shahi in the 16th century. Reviewing the longue durée idea of Iranian kingship perceived within the Safavid royal ideology in the reigns of the first two monarchs, we delve into the Shahnama paintings to see the artists’ assets for showing the divinity and dignity of Iranian kings and how they managed to mark a difference between the profane and Shiite iconographies. We took as case studies nine illustrations of Zahhak’s story, the most tyrannical legendary King in Shah Tahmasp’s Shahnama. Indeed, to analyse the artistic manifestations of the Royal farr, we examined the scenes where the “true” king is literally absent.

Reinvention of Padishah-i Islam in the Visual Representations of Ghazan Khan

This article discusses how the visualization of Mahmud Ghazan, the sixth Ilkhanid ruler, was employed to construct and propagate his image as the Padishah-i Islam (King of Islam), thus justifying him both as king of Iranshahr (land of Iran) and the legitimate successor of the Prophet Muhammad. In a quest for visual translations of the Ilkhanid concept of Padishah-i Islam—an inseparable combination of the Persian notion of ideal kingship and prophethood—several illustrations from the Diez albums representing Ghazan or events of his reign have been identified, two of which have become subject to detailed iconographic analyses. Two approaches to the visualization of Ghazan as Padishah-i Islam can be considered here. The first is Ghazan’s birth scene where the visual narrative is covered in various formal and semantic layers; transformed into a symbolic narrative of a holy birth associated with those of the prophets. The second appears in the scene of Ghazan’s enthronement, probably once illustrated as an unknown manuscript’s frontispiece, the images’ composition and components appear as a visual panegyric poem which applies an elaborate visual language that elevates Ghazan to the level of a divine king.

Reinvention of Padishah-i Islam in the Visual Representations of Ghazan Khan

This article discusses how the visualization of Mahmud Ghazan, the sixth Ilkhanid ruler, was employed to construct and propagate his image as the Padishah-i Islam (King of Islam), thus justifying him both as king of Iranshahr (land of Iran) and the legitimate successor of the Prophet Muhammad. In a quest for visual translations of the Ilkhanid concept of Padishah-i Islam—an inseparable combination of the Persian notion of ideal kingship and prophethood—several illustrations from the Diez albums representing Ghazan or events of his reign have been identified, two of which have become subject to detailed iconographic analyses. Two approaches to the visualization of Ghazan as Padishah-i Islam can be considered here. The first is Ghazan’s birth scene where the visual narrative is covered in various formal and semantic layers; transformed into a symbolic narrative of a holy birth associated with those of the prophets. The second appears in the scene of Ghazan’s enthronement, probably once illustrated as an unknown manuscript’s frontispiece, the images’ composition and components appear as a visual panegyric poem which applies an elaborate visual language that elevates Ghazan to the level of a divine king.

Interlingual live subtitling: the crossroads between translation, interpreting and accessibility

AbstractWhereas prototypical conceptions of translation and interpreting as two separate and distinct entities are still being upheld in different contexts, the professional reality shows a less clear-cut picture. Given the relentless advancement of technological innovation and the important role played by (media) accessibility in society, translation, interpreting and accessibility are in a state of flux, with increasingly blurring boundaries between them. Interpreting is not only concerned with spoken and signed modalities, but also with written target texts, which links it to audio–visual translation. In turn, audio–visual translation is no longer only about dubbing and subtitling films from one language to another. It now covers the provision of intra- and interlingual translation for live programmes and live events for users who have no access to the original version (be it for linguistic, sensory-related or other reasons). Speech-to-text interpreting (STTI), also referred to as live subtitling, is a communication-enabling service that allows the production of written access to live events or programmes for people with and without hearing loss. It thus brings together translation, interpreting and accessibility. The aim of this paper is to introduce STTI for readers who may not be familiar with it and to delve into interlingual live subtitling (ILS), and more specifically to explore the research carried out to date about this new technique, its professional development and the way in which it is being trained.

The Magic Circle: Visually Representing the Woman of Endor (ovot)

The interaction between Saul and the woman of Endor is a minor episode, not only within the first book of Samuel, but in the Tanakh as a whole. The woman, whose actions drive the narrative, is not named nor is very much known about her. All that the text reveals are her gender, domicile location, familiarity with the law (Saul’s ban on ovot and yid’onim), and culinary possessions (flour and a calf). Through linguistic connotation her age range is surmisable: she is neither maiden nor elderly. Of her specific abilities, little is actually known as it is totally unclear what she did exactly that caused connection between the living Saul and deceased Samuel. Yet this woman, and the language used that describes her in a few verses without substantive information, have singularly driven wildly polarizing visual representations of the female thaumaturge for centuries. The artistically rendered visage of the Endoran ranges from the crone of gothic nightmares to hypersexual femme fatale, to gregarious earth mother, to occult adept perfecting her craft. Despite her shifting guise, one aspect remains consistent throughout the majority of images: the visual translation (or mistranslation) of ovot.&#x0D;

“Visual Translation and the Amazing Broken Telephone Kaleidoscope”: A Dialogue

Kat Germain considers media accessibility strategies while translating signed music by Deaf artists, drawing on her experience as a visual translator/interpreter and audio describer for Blind and partially sighted audience members. Germain queries: What is the disparity between a Deaf artist’s (nonvocal) work and a Blind audience member’s experience of it? When does visual translation stop being a copy, and when does it become a work of art in its own right if the audience member’s only experience is the audible translation, an unavoidably a creative act? Along with Germain, the panel (Deaf scholar/artist, Blind scholar/creator/audience member, and a sound engineer) discusses aspects of ethics and aesthetics of accessibility in the arts.

Performativity and intersemiotic translation in contemporary art: the case of Hong Kong Atlas

This paper considers the intersection of performativity and intersemiotic translation in contemporary art through a case study of a new media art project aimed at visually transcoding Dung Kai-Cheung’s novel Atlas: The Archaeology of an Imaginary City, a book of postmodern fiction about the palimpsest nature of Hong Kong as a linguistic landscape and a city of (cultural) translation. In Hong Kong Atlas, the locations in Dung’s book are performatively mapped out onto the real semioscape of contemporary Hong Kong using psychogeography documented in digital images, which are then transcoded through a series of iterative translations into a variety of visual formats. By analysing the complex methodology and the unique interdisciplinary theoretical framework underpinning this artistic research (combining insights from fields such as visual studies, translation studies, sociolinguistics, multimodal discourse analysis, art theory, and practice-based research epistemology), the article aims to provide a novel approach to the discussion of visual translation as intersemiotic translation.

I Dream of Being a Beautiful Princess but I Am a Socialist Girl: The Psychological and Ideological Aspects of the Illustrations for the Fairy Tale “Cinderella” in Bulgaria in the Second Half of the 20th Century

Fairy tales are often accompanied by illustrations that extend and complicate the messages of the text, and adapt it to the specific characteristics of different political and cultural situations. This article focuses on the images of one of the best-known fairy tales recorded by Charles Perrault and the Brothers Grimm, “Cinderella,” and looks for an answer to the question of how, in the second half of the 20th century, when the world was divided by the Iron Curtain, the socialist ideology attempted to make a “visual translation” of the story, which had been known for centuries, thus sending new aesthetic and political messages to adolescents. The emphasis is placed not only on the opposition of the roles of woman in socialist and capitalist societies but also on making the differences in appearance, behavior, and upbringing stand out.

Visual Translation: Illuminated Manuscripts and the First French Humanists by Anne D. Hedeman

Reviewed by: Visual Translation: Illuminated Manuscripts and the First French Humanists by Anne D. Hedeman Anne Rochebouet Visual Translation: Illuminated Manuscripts and the First French Humanists. By Anne D. Hedeman. University of Notre Dame Press, 2022. Anne D. Hedeman's Visual Translation, adapted from a series of lectures at the University of Notre Dame in 2013, arises from her extensive research on early French humanism and its material reception through the study of humanist book production. In this very engaging and abundantly illustrated book, she, in part, draws and expands on previously published material, creating a cohesive, convincing study of lavishly illuminated manuscripts, that were produced for courtly audiences but were closely associated with two well-known figures of French humanism in the early fifteenth century, Laurent de Premierfait and Jean Lebègue. Chapter 1 begins with a brief presentation on the Parisian humanist network of the early fifteenth century and the known involvement of Premierfait and Lebègue in book production. The chapter puts into perspective the role they played as voluntary intermediaries between, on the one hand, Latin classics (Cicero, Sallust, Statius, Terence) or Italian contemporary literature (Boccaccio) and, on the other hand, their contemporaneous princely audience. Hedeman argues that the two humanists became gradually aware of the way in which images and, more generally, the manuscripts' mise-en-page and mise-en-texte could be a crucial aid in understanding and interpreting their content (a process she calls, following Claire Richter Sherman, visual translation). This held true particularly for texts that were culturally and/or chronologically removed from the environment in which they were originally received. Hedeman's study considers how Premierfait and Lebègue shaped access to these texts for a French audience that was used to lavishly illustrated books, and her early chapters focus on their links to manuscripts of Latin classics. Chapter 2 examines Premierfait's involvement in the production, between 1405 and 1407, of two luxurious manuscripts copied by the same scribe, who also worked for him. The first—London, British Library, Burney MS. 257—contains 129 illuminations illustrating Statius's Thebaid and Achilleid. The second—a copy of Terence's [End Page 144] Comedies decorated with 142 illuminations (Paris, Bibliothèque nationale de France, lat. 7907A)—was a New Year's gift from Martin Gouge to John of Berry in 1408. Although these manuscripts contain well-known Latin texts (school classics that were also favorites in humanist circles), they were, at this time, notably absent from aristocratic libraries. Courtly audiences were familiar with the stories of Thebes and Troy but primarily through French adaptations, such as the Histoire ancienne jusqu'à César and the Romans d'antiquité. As Hedeman points out, the two manuscripts studied here had both to invent entirely new visual cycles. Earlier copies of Statius were mostly devoid of illustrations; the works of Terence, in contrast, had a rich iconographic tradition since the Carolingian period, mostly inspired, then, by copies made in late antiquity, but these images were not designed specifically for a courtly audience. Hedeman studies these cycles in detail, always in close relation with the text they mediate. She shows how the pictures are an integral part of the critical apparatus that is intended to facilitate readers' access to the Latin text. The Statius volume, for example, contains no rubrics; but the illuminations, which open all but one chapter of the Achilleid and Thebaid, give readers a structure through which to apprehend the text, alongside the chapter summaries in Latin prose that close the volume. The images, in Terence's Comedies, are placed at the beginning of each scene and play on medieval costumes and contemporaneous artistic conventions to identify the characters' roles and social statuses. By visualizing the classical texts through a fifteenth-century frame of reference, the manuscripts work to make the past present and to facilitate readers' understanding. Chapter 3 focuses on Lebègue's role in the development of a large visual cycle for Sallust's Conspiracy of Catiline and Jugurthine War. (Other densely illustrated copies of Sallust did not exist at the beginning of the fifteenth century.) In addition to these manuscripts, Hedeman considers a unique set of French directions on how...

Visual Translation, Design, and Language Justice: a Case Study from North Central Florida

Purpose: This article presents a case study of how a bilingual technical communicator and a bilingual visual designer collaborated to visualize stories of language access in North Central Florida. Method: We combined participatory methods used in technical communication (Agboka, 2013; Rose and Cardinal, 2018) with interviews and design methods (Gonzalez Viveros et al., 2020) from both technical communication and visual design. The goal of the study was to document the languages spoken by immigrant community members in North Central Florida, to interview immigrant community members about their languages, and to transform interview data into visual designs that could inspire conversation about language justice in our community. Results: By transforming interview data into visual designs (i. e., collages), we were able to understand the complexity that language plays in the lives of multilingual communities, gaining insights into both the challenges and the advantages of speaking multiple languages as immigrants in North Central Florida. Conclusion: We encourage other technical communication and design researchers to implement visual data approaches in their work, particularly when working with participants whose language histories span beyond white American Englishes. As technical communication continues expanding into more global contexts and as language diversity continues to be a reality in contemporary technical communication work, interdisciplinary collaborations among technical communicators, translators, and designers, will continue to gain importance and impact, particularly in community- driven projects.