Speech Translation Research Articles

WTASR: Wavelet Transformer for Automatic Speech Recognition of Indian Languages

Automatic speech recognition systems are developed for translating the speech signals into the corresponding text representation. This translation is used in a variety of applications like voice enabled commands, assistive devices and bots, etc. There is a significant lack of efficient technology for Indian languages. In this paper, an wavelet transformer for automatic speech recognition (WTASR) of Indian language is proposed. The speech signals suffer from the problem of high and low frequency over different times due to variation in speech of the speaker. Thus, wavelets enable the network to analyze the signal in multiscale. The wavelet decomposition of the signal is fed in the network for generating the text. The transformer network comprises an encoder decoder system for speech translation. The model is trained on Indian language dataset for translation of speech into corresponding text. The proposed method is compared with other state of the art methods. The results show that the proposed WTASR has a low word error rate and can be used for effective speech recognition for Indian language.

Introduction to Research and Development Activities of Advanced Speech Translation in NICT

Introduction to Research and Development Activities of Advanced Speech Translation in NICT

The Analysis of Imperative Speech Acts of Translation Strategy in Subtitle Serial Squid Game (Episode 1)

Lately, broadcasts from South Korea have been in great demand in the world, including Indonesia. These shows can be in the form of videos, films, serials and dramas. A presentation displays the behavior or action conveyed through communication between the speaker (speaker) and the listener (opposite), which is referred to as a speech act. One type of speech act is imperative speech act. An imperative speech act is a form of speech in which the speaker demands action from the listener. To understand the broadcast, it through the translation results which are displayed in the form of subtitles. There are many translation strategies that can be used, not only literal strategies, to do the translation. This study aims 1) to reveal translation strategies used to translate imperative utterances in Squid Game episode 1, and 2) to analyze the types of translation strategies used to translate imperative utterances in Squid Game episode 1 dialogue series. This research used descriptive qualitative method. Speech data and translation results were collected through the Language Reactor and analyzed using Salihen Moentaha’s translation strategy theory. The results showed that there were 53 direct imperative utterances found. From the translation of the speech, 8 types of translation strategies were found, namely Literal (23 times), Free Translation (6 times), Paraphrasing (12 times), Replacement (3 times) with word class types (1 time) and replacement of part of the sentence with active operations types. -passive (2 times), Lexical replacement (12 times) with the type of concretization (2 times); Generalization (3 times); Antonym (2 times); Compensation (2 times), Addition (17 Times), Omission (8 Times), Compression (27 Times), Explication (2 Times). The most dominant translation strategy used is the Compression strategy.

Grammatical transformations in translation of political speeches

The paper is devoted to the problem of translation of political speeches, which is extremely relevant today, as political activity plays a special role in society, affects the country’s place in the international arena and its relations with other coun-tries. Politicians have the opportunity to appeal to both the international community and their compatriots, gain recognition, demonstrate their leadership and represent their country in the best possible way via speeches. Translation of political speeches is considered to be one of the most complicated types of translation, which is the area of increased linguistic responsibility. Translation transformations, which are defined as certain structural or semantic changes that are the result of lexical and grammatical differences in languages, are used to achieve translation equivalence (translation adequacy) despite the differences in the formal and semantic systems of two languages. The research is aimed at examining the peculiarities of political speeches and difficulties arising in the process of their rendering into Ukrainian. This paper consists of six main sections. The first is an introduction to the study. The second examines the definition and features of political speeches given by national and foreign scientists. The third section explains the methodology used in the study and the data selection process. In the course of analysis, the existing scientific literature concerning the topic at hand providing valid grounds for the practical part of the work has been analyzed and grammatical translation transformation examples have been selected and examined. The research data was taken from the website of the US Embassy in Ukraine. Thereafter the focus shifts to challenges the students face in translation of political speeches and highlights the grammatical transformations classification that constituted the theoretical basis of the research. The fifth section is devoted to analyzing the data and the last section summarizes the findings of the study.

Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning

The end-to-end speech translation (E2E-ST) model has gradually become a mainstream paradigm due to its low latency and less error propagation. However, it is non-trivial to train such a model well due to the task complexity and data scarcity. The speech-and-text modality differences result in the E2E-ST model performance usually inferior to the corresponding machine translation (MT) model. Based on the above observation, existing methods often use sharingmechanisms to carry out implicit knowledge transfer by imposing various constraints. However, the final model often performs worse on the MT task than the MT model trained alone, which means that the knowledge transfer ability of this method is also limited. To deal with these problems, we propose the FCCL (Fine- and Coarse- Granularity Contrastive Learning) approach for E2E-ST, which makes explicit knowledge transfer through cross-modal multi-grained contrastive learning. A key ingredient of our approach is applying contrastive learning at both sentence- and frame-level to give the comprehensive guide for extracting speech representations containing rich semantic information.In addition, we adopt a simple whitening method to alleviate the representation degeneration in the MT model, which adversely affects contrast learning. Experiments on the MuST-C benchmark show that our proposed approach significantly outperforms the state-of-the-art E2E-ST baselines on all eight language pairs. Further analysis indicates that FCCL can free up its capacity from learning grammatical structure information and force more layers to learn semantic information.

PECULIARITIES OF REPRODUCING EXPRESSIVE MEANS IN POLITICAL SPEECHES TRASLATION

The translation of political speeches is crucial in today's globalized world, where leaders communicate with audiences of different languages and cultures. Reproducing expressive devices is particularly important as it significantly affects the understanding and interpretation of the message. This study aims to identify the peculiarities of reproducing expressive means in political speeches translation, including metaphors, allegories, antitheses, and other rhetorical devices used by prominent politicians.The primary goal is to ensure that the translated speech accurately conveys the intended message and style of the speaker while adapting them to the cultural and linguistic context of the target audience. Materials and methods of research involved analyzing speeches of famous politicians in the source language, working with native speakers of the target language to ensure accuracy and cultural sensitivity and reviewing the translated speech for clarity, coherence, and persuasiveness.The study's results indicate that the translation of expressive means requires careful consideration of cultural, linguistic, and rhetorical contexts, and appropriate target language equivalents to maintain the integrity of the original message and style. The understanding of these nuances can enhance cross-cultural communication and foster global cooperation. In conclusion, the study highlights the importance of reproducing expressive means in political speeches translation to ensure clear, coherent, and persuasive communication across languages and cultures.

ПРОБЛЕМА УТРАТЫ ЭМОТИВНОСТИ ПРИ ПЕРЕВОДЕ (НА МАТЕРИАЛЕ ПЕРЕВОДА ОФИЦИАЛЬНЫХ ВЫСТУПЛЕНИЙ ПОСТОЯННЫХ ПРЕДСТАВИТЕЛЕЙ РОССИЙСКОЙ ФЕДЕРАЦИИ В ОРГАНИЗАЦИИ ОБЪЕДИНЕННЫХ НАЦИЙ)

The article deals with the problem of the open professional diplomatic discourse emotivity, the sentiment of speeches within the UN Security Council, the problems of the sentiment changing during the official translation of speeches from Russian into English. The relevance is determined by the need to analyze the emotivity transfer degree of speeches in the translation process, because texts of speeches translated into one of the official languages of the United Nations – English – due to its status as an international language are more accessible to readers and because of this are able to have a more significant impact on the audience. The purpose is to analyze the results of translation of spoken texts by means of linguistic and computer analysis to identify changes in the level of emotivity of the source and translated texts. Reasons for the change in tonality during translation are discussed.

ELT in the Age of Artificial Intelligence (AI): Working with Machines

Since 2016, the development of artificial intelligence (AI) has been strong and pervasive, including its roles in business and education. In ELT, in particular, several machine learning models have been implemented such as speech recognition, grammar correction, chatbots, and translation. ELT is in the middle of a rapid and disruptive change and the magnitude of which is paramount that we have never witnessed before. Under this situation, ELT practitioners may have to acquire additional skills and competencies so as to be relevant and thrive in this rapid change and move the field forward to the next level. In this paper, the researcher proposes ELT 3.0, a new vision where working with the machines needs to be incorporated into the existing roles of the ELT teachers.

Speech translation for Unwritten language using intermediate representation: Experiment for Viet-Muong language pair

The paper studies an automatic translation method that translates from the text of a language (L1) to the speech of an unwritten language (L2). Normally the written text is used as the bridge to connect a translation module that translates from the text of L1 to the text of L2 and a synthesis module that generates the speech of L2 from the text. In the case of unwritten language, an intermediate representation has to be used instead of the writing form of L2. This paper proposes the use of phoneme representation because of the intimate relationship between phonemes and speech in one language. The proposed method was applied to the Viet-Muong language pair. The Vietnamese text needs to be translated into Muong language in two dialects, Muong Bi - Hoa Binh and Muong Tan Son - Phu Tho, both unwritten. The paper also proposes a phoneme set for each Muong language and applies them to the problem. The evaluation results showed that the translation quality was relatively high in both dialects (for Muong Bi, the fluency score was 4.63/5.0, and the adequacy score was 4.56/5.0). The synthesized speaking quality in both dialects is acceptable (for Muong Bi, the MOS score was 4.47/5.0, and the comprehension score was 93.55%). The results also show that the applicability of the proposed system to other unwritten languages ​​is promising.

The Unfolding Illiberalism in Hungary

Abstract The emerging literature on illiberalisms offers a framework for detecting whether or not a special Hungarian illiberalism has been unfolding. Over the last 12 years, Hungary’s former liberal constitutionalist nature has been changed to illiberal constitutionalism. This transformation of the constitutional system recently culminated with the fourth consecutive electoral victory of Viktor Orbán in April 2022. Nevertheless, the possible ideological nature of Orbán’s regime has been understudied so far. Drawing on some newly published literature on illiberalisms, I claim that changes in electoral law (2014), the revised approach to the relationship between EU law and domestic law (2016 and 2021), and identity rights (2013, 2019–2020), along with the preceding political narratives, can be viewed as seeds from which a special, state-driven illiberalism has been growing in Hungary. If it is so, it could be concluded that Hungarian illiberalism is based on and the result of two factors: i) the translation of illiberal speech into the language of the law, and ii) the creation of intertwined identities for the constitution, the state, and citizens, which are based on (ethno)nationalism, sovereigntism, traditionalism and heteronormativity against mainly EU obligations and Western values.

Information-Theoretic Method for Assessing the Quality of Translations.

In recent years, the task of translating from one language to another has attracted wide attention from researchers due to numerous practical uses, ranging from the translation of various texts and speeches, including the so-called "machine" translation, to the dubbing of films and numerous other video materials. To study this problem, we propose to use the information-theoretic method for assessing the quality of translations. We based our approach on the classification of sources of text variability proposed by A.N. Kolmogorov: information content, form, and unconscious author's style. It is clear that the unconscious "author's" style is influenced by the translator. So researchers need special methods to determine how accurately the author's style is conveyed, because it, in a sense, determines the quality of the translation. In this paper, we propose a method that allows us to estimate the quality of translation from different translators. The method is used to study translations of classical English-language works into Russian and, conversely, Russian classics into English. We successfully used this method to determine the attribution of literary texts.

Bridging the cross-modal gap using adversarial training for speech-to-text translation

The end-to-end speech translation (ST) model usually adopts the encoder-decoder structure, which takes the speech of the source language as input and directly outputs its translation result in the target language. Since the model performs cross-modal translation, it needs to extract semantically rich representations from speech. Compared with text, speech is fine-grained and contains more noise, which puts a great burden on the encoder of the ST model. The modal gap between speech and text usually causes the ST model to perform inferior to the corresponding machine translation (MT) model. To bridge the cross-modal gap, this paper proposes to use adversarial training to relieve the burden on the ST encoder by providing internal supervision signals. In this approach, the encoder in the ST model can extract representations that contain rich semantics, which greatly improves performance. Experiments on the datasets of Augmented Librispeech English-French and MuST-C English-German show the effectiveness of our method. Further analysis indicates that our proposed method can perform well in low-resource conditions compared to strong baselines.

SAMU-XLSR: Semantically-Aligned Multimodal Utterance-Level Cross-Lingual Speech Representation

We propose the SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation learning framework. Unlike previous works on speech representation learning, which learns multilingual contextual speech embedding at the resolution of an acoustic frame (10-20ms), this work focuses on learning multimodal (speech-text) multilingual speech embedding at the resolution of a sentence (5-10s) such that the embedding vector space is semantically aligned across different languages. We combine state-of-the-art multilingual acoustic frame-level speech representation learning model XLS-R with the Language Agnostic BERT Sentence Embedding (LaBSE) model to create an utterance-level multimodal multilingual speech encoder SAMU-XLSR. Although we train SAMU-XLSR with only multilingual transcribed speech data, cross-lingual speech-text and speech-speech associations emerge in its learned representation space. To substantiate our claims, we use SAMU-XLSR speech encoder in combination with a pre-trained LaBSE text sentence encoder for cross-lingual speech-to-text translation retrieval, and SAMU-XLSR alone for cross-lingual speech-to-speech translation retrieval. We highlight these applications by performing several cross-lingual text and speech translation retrieval tasks across several datasets.

Conceptual Metaphors in the Translation of Political Speeches of King Abdullah II

The research aims to define conceptual metaphor and its types in general, as well as exploring the conceptualization of the metaphorical expressions used in the political speeches of King Abdullah II in English and the translations of the metaphors in Arabic. The study illustrates the techniques and strategies used by the Hashemite Royal Court in translating ST metaphorical expression to the TT. The researcher relies on the Metaphor Identification Procedure (MIP), proposed by the Group (2007) and the Critical Metaphor Analysis (CMA), proposed by Charteris-Black (2004), in identifying the conceptual metaphors that King Abdullah II used in his political speeches. As for categorizing the metaphorical expressions, we rely on the Longman and Oxford Dictionaries to determine the semantic field that the metaphorical expression relates to. The study depends on the method proposed by Al-Zoubi, Al-Ali, and Al-Hasnawi (2007) to examine the conceptual metaphors of the two texts and find out whether there are any differences or similarities by comparing the two metaphors, as well as analyzing whether the metaphors are maintained, changed or deleted.

Design of Intelligent Speech Translation System Based on Deep Learning

In order to solve the problem of low translation accuracy caused by complex sentence parameters in traditional machine translation systems, a method based on deep learning was proposed. First, MCU SPCE061A is used to study the problem of complex digital signal. The training data in the synchronous translation server support the translation services of a large number of users, and the translation results were displayed through the session interface of the user terminal. The PMDL model is used to detect the keyword signal, record the PCM audio data, and slice the collected pulse code modulation signal, so as to wake up the artificial intelligence voice service. Then, this study establishes a speech recognition process that accurately outputs the speech-related semantics. In this paper, a manual interactive synchronous translation program is designed with the input text as the search criterion, and the set is trimmed to obtain the best translation effect. The experimental results show that the sentence translation accuracy of the system is 0.9 ∼ 1.0. It is proved that the method based on deep learning solves the problem of low accuracy of the traditional translation system.

"Scandalous Speech and Slanderous Libelles": Robert Peterson, Claudio Tolomei, and the Translation of Free Speech in Early Modern England

"Scandalous Speech and Slanderous Libelles": Robert Peterson, Claudio Tolomei, and the Translation of Free Speech in Early Modern England

Application of Discriminative Training Algorithm Based on the Improved Gaussian Mixture Model in English Translation Evaluation

English translation, also a kind of language conversion, refers to the activities of expressing each other in English to express another language or another language to express English. This paper aimed to evaluate and research methods of English translation based on the improved Gaussian mixture model (GMM). The paper proposed a simple evaluation and analysis of English translation using a discriminative training algorithm. In the experimental part, on the one hand, taking students, teachers, and translators as case study objects, through a questionnaire survey, it can be known that the number of students who scored more than 80 points in the English translation test is increasing every year, from 11.30% in the first year to 21.13% in the fourth year, an increase of nearly 10%. On the other hand, it can be seen from the speech translation experiment that the correct translation rate was 46.18% by using intelligent technology to distinguish translation. The experimental results showed that the discriminative training algorithm under the improved GMM algorithm is effective in the research of English translation evaluation.

Efficient optical reservoir computing for parallel data processing

We propose and experimentally demonstrate an optical reservoir computing system in free space, using second-harmonic generation for nonlinear kernel functions and a scattering medium to enhance reservoir nodes interconnection. We test it for one-step and multi-step predication of Mackey-Glass time series with different input-mapping methods on a spatial light modulator. For one-step prediction, we achieve 1.8 × 10-3 normalized mean squared error (NMSE). For the multi-step prediction, we explore two different mapping methods: linear-combination and concatenation, achieving 16-step prediction with NMSE as low as 3.5 × 10-4. Robust and superior for multi-step prediction, our approach and design have potential for parallel data processing tasks such as video prediction, speech translation, and so on.

Pragmatic Equivalence of Expressive Speech Act in Television Series Bridgerton

This article aims to identify pragmatic equivalence in the translation of expressive speech acts. Descriptive and comparative methods were used in this study to analyze the data. Data collection is done by analyzing documents with the technique of observing and note-taking. The data is in the form of utterances containing expressive speech acts on the subtitles of the Bridgerton TV series in English and Indonesian. The data is obtained through accessing the Netflix platform with a personal account. Found 123 data of expressive speech acts on the Bridgerton TV series subtitle with the functions of criticizing, praising, sarcastic, complaining, apologizing, condolences and thanks. The result of the research shows that there are two types of pragmatic equivalence that the Bridgerton TV series subtitle translator has successfully achieved in translating expressive speech acts. The equivalences are the equivalence of expressive illocutionary force and the equivalence of the distance of relevance. Thus, the translator has been able to convey messages from the source text to the target text commensurately.

Direct Speech-to-Text Translation Models as Students of Text-to-Text Models

Direct speech-to-text translation (ST) is an emerging approach that consists in performing the ST task with a single neural model. Although this paradigm comes with the promise to outperform the traditional pipeline systems, its rise is still limited by the paucity of speech-translation paired corpora compared to the large amount of speech-transcript and parallel bilingual corpora available to train previous solutions. As such, the research community focused on techniques to transfer knowledge from automatic speech recognition (ASR) and machine translation (MT) models trained on huge datasets. In this paper, we extend and integrate our recent work (Gaido, Gangi, et al. 2020) analysing the best performing approach to transfer learning from MT, which is represented by knowledge distillation (KD) in sequence-to-sequence models. After the comparison of the different KD methods to understand which one is the most effective, we extend our previous analysis of the effects – both in terms of benefits and drawbacks – to different language pairs in high-resource conditions, ensuring the generalisability of our findings. Altogether, these extensions complement and complete our investigation on KD for speech translation leading to the following overall findings: i) the best training recipe involves a word-level KD training followed by a fine-tuning step on the ST task, ii) word-level KD from MT can be detrimental for gender translation and can lead to output truncation (though these problems are alleviated by the fine-tuning on the ST task), and iii) the quality of the ST student model strongly depends on the quality of the MT teacher model, although the correlation is not linear.