Machine Translation Research Articles

Translating the Untranslatable: DeepL and ChatGPT on Academic Idioms

This research explores the efficacy of two prominent machine translation platforms, DeepL and ChatGPT, in translating academic idioms from English to Indonesian. Academic idioms, situated between discipline-specific jargon and universally understood expressions, pose a challenge for existing translation systems, particularly those rooted in Neural Machine Translation (NMT). The study employs a qualitative descriptive methodology, focusing on translation precision and naturalness, with bilingual experts evaluating translations through a questionnaire, focusing on translation precision and naturalness. The comprehensive analysis involved 50 participants who assessed translations on a scale of accuracy and fluency using Fiederer and O'Brian's (2009) rating scale. The results indicate that both platforms exhibit strengths and weaknesses in terms of accuracy and fluency. While DeepL demonstrates trust in its translation proficiency, ChatGPT receives a more favorable response, especially regarding fluency. Participants preferred ChatGPT for fluency in handling academic expressions, indicating its adaptability. The study also revealed a general agreement among participants regarding the difficulties both platforms encounter in accurately translating academic idioms, emphasizing continuous requirements for improved machine translation. These insights enhance understanding of machine translation's strengths and limitations in academic setting, with implications for future technology development.

Students’ Multilingual Experiences With Online Machine Translation: Implications for General Language Use

This article discusses how online machine translators (OMT) can influence the way people use general language or languages. The latter distinction has to do with the fact that contemporary lifestyles turn language users into plurilingual communicators. Online shopping, travelling abroad or social media often require at least some degree of plurilingual text mediation skills. Irrespective of foreign language competences, a lot of plurilingual communicators seek OMT support in making sure their messaging with international partners is correct and effective. The article discusses research results concerning how a selected group of BA and MA students in education use machine translation for day-to-day plurilingual communication tasks. We asked them about the relation between machine translation, language learning and language use. We give some insight into the machine translation engine DeepL Translator, which can effectively be used in support of plurilingual text mediation. The article also contains basic educational implications for why and how to make OMT part of foreign language curricula.

Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets

Abstract Recent machine translation (MT) metrics calibrate their effectiveness by correlating with human judgement. However, these results are often obtained by averaging predictions across large test sets without any insights into the strengths and weaknesses of these metrics across different error types. Challenge sets are used to probe specific dimensions of metric behaviour but there are very few such datasets and they either focus on a limited number of phenomena or a limited number of language pairs. We introduce ACES, a contrastive challenge set spanning 146 language pairs, aimed at discovering whether metrics can identify 68 translation accuracy errors. These phenomena range from basic alterations at the word/character level to more intricate errors based on discourse and real-world knowledge. We conducted a large-scale study by benchmarking ACES on 47 metrics submitted to the WMT 2022 and WMT 2023 metrics shared tasks. We also measure their sensitivity to a range of linguistic phenomena. We further investigate claims that Large Language Models (LLMs) are effective as MT evaluators, addressing the limitations of previous studies by using a dataset that covers a range of linguistic phenomena and language pairs and includes both low- and medium-resource languages. Our results demonstrate that different metric families struggle with different phenomena and that LLM-based methods are unreliable. We expose a number of major flaws with existing methods: most metrics ignore the source sentence; metrics tend to prefer surface level overlap; and over-reliance on language-agnostic representations leads to confusion when the target language is similar to the source language. To further encourage detailed evaluation beyond singular scores, we expand ACES to include error span annotations, denoted as SPAN-ACES and we use this dataset to evaluate span-based error metrics, showing that these metrics also need considerable improvement. Based on our observations, we provide a set of recommendations for building better MT metrics, including focusing on error labels instead of scores, ensembling, designing metrics to explicitly focus on the source sentence, focusing on semantic content rather than relying on the lexical overlap, and choosing the right pre-trained model for obtaining representations.

Cross-Lingual Short-Text Semantic Similarity for Kannada–English Language Pair

Analyzing the semantic similarity of cross-lingual texts is a crucial part of natural language processing (NLP). The computation of semantic similarity is essential for a variety of tasks such as evaluating machine translation systems, quality checking human translation, information retrieval, plagiarism checks, etc. In this paper, we propose a method for measuring the semantic similarity of Kannada–English sentence pairs that uses embedding space alignment, lexical decomposition, word order, and a convolutional neural network. The proposed method achieves a maximum correlation of 83% with human annotations. Experiments on semantic matching and retrieval tasks resulted in promising results in terms of precision and recall.

Research on English–Chinese machine translation shift based on word vector similarity

Research on English–Chinese machine translation shift based on word vector similarity

Risk perception and risk management in legal translation: a questionnaire survey

ABSTRACT While risk as a concept has aroused interest in Translation Studies in recent years, empirical research on the topic is still lacking. Mostly regarded as an adverse factor, risk has close conceptual links to translation quality and translation ethics. Legal translation is a field in which risk awareness and risk management are of the utmost importance given the values at stake. Curiously, the number of lawsuits or formal complaints against translators is very low. A questionnaire survey was conducted among Czech legal translators to provide insights into their perception of risk and risk management methods. The results show that legal translation is mostly regarded as riskier than other areas of translation and that risk perception partly depends on previous exposure to the field. Translating into a foreign language, though potentially risky, is almost unanimously considered inevitable. Literal equivalents are perceived as safer than loose equivalents, but loose equivalents tend to be preferred with increasing practice. Complaints from clients are extremely rare and never have serious consequences. Current developments in the industry, such as the increasing use of internet resources and machine translation, are likely to have an impact on risk perception and risk management in the future.

An Attempt to Generate Mozart's Piano Compositions Based on an LSTM Model

LSTM, proposed by German computer scientists Sepp Hochreiter and Jürgen Schmidhuber in 1997 [3], has shown outstanding performance in various tasks, especially in tasks that require capturing long-term dependencies. These tasks include language modeling and machine translation in the field of natural language processing, as well as stock price prediction. This project aims to explore the potential of LSTM models in the field of music generation by analyzing Mozart's classical piano works using music21 and creating a training dataset. The project uses LSTM to learn this dataset, adjusting the forget gate, input gate, and different training iteration times to generate diversified outputs. Finally, the generated music data was manually selected and labeled.

Initial exploration into sarcasm and irony through machine translation

In this paper, we investigate sarcasm and irony as seen through a novel perspective of machine translation. We employ various techniques for translation, comparing both manually and automatically translated datasets of irony and sarcasm. We first clarify the definitions of irony and sarcasm and present an exhaustive field review of studies on irony both from purely linguistic as well as computational linguistic perspectives. We also propose a novel evaluation metric for the purpose of evaluating translations of figurative language, with a focus on machine-translated irony and sarcasm. The constructed English and Chinese parallel dataset includes polarized content from tweets as well as forum posts, categorized by irony types. The preferred translation model, mBART-50, is identified through a thorough experimental process. Optimal translation settings and the best-finetuned model for irony are explored, with the most effective model being finetuned on both ironic and non-ironic data. We also experimented which types of irony are best suitable for training in this specific task — short microblogging messages or longer forum posts. Moreover, we compare the capabilities of a well fine-tuned mBART to a prompt-based method using the recently popular ChatGPT model, with the conclusion that the former still outperforms the latter, although ChatGPT without any training can be considered as a “good enough” ad hoc solution in the case of a lack of data for training. Finally, we verify if the translated data – either manually, or with an MT model – can be used as training data in a task of irony detection. We believe that the presented research can be expanded into languages other than the presented here Chinese and English, which together with the ability to detect various categories of irony, could contribute to deepening the understanding of figurative language, especially irony and sarcasm.

Balancing quality and efficiency: An improved non-autoregressive model for pseudocode-to-code conversion

Pseudocode can efficiently represent algorithm logic, but manual conversion to executable code requires more time. Recent works have applied autoregressive (AR) models to automate pseudocode-to-code conversion, achieving good results but slow generation speed. Non-autoregressive (NAR) models offer the advantage of parallel generation. However, they face challenges in effectively capturing contextual information, leading to a potential degradation in the quality of the generated output. This paper presents an improved NAR model for balancing quality and efficiency in pseudocode conversion. Firstly, two strategies are proposed to address out-of-vocabulary and repetition problems. Secondly, an improved NAR model is built using linear smoothing and adaptive techniques in the transition matrix, which can mitigate the “winner takes all” effect. Finally, a new synthesis potential metric is proposed for evaluating pseudocode conversion. Experimental results show that the proposed method matches AR model performance while accelerating generation over 10-fold. Further, the proposed NAR model reduces the gap with the AR model in terms of the BLEU score on the EN-DE and DE-EN tasks of the WMT14 machine translation.

A Comparative Study on the Quality of English-Chinese Translation of Legal Texts Between ChatGPT and Neural Machine Translation Systems

A Comparative Study on the Quality of English-Chinese Translation of Legal Texts Between ChatGPT and Neural Machine Translation Systems

Human-Machine Interaction Translation under Artificial Intelligence and Big Data: Analysis from the Perspective of Text Stratification and Corpus Construction

With the continuous advancement of artificial intelligence and big data technologies, the neural network-based machine translation driven by deep learning continues to flourish. This ongoing progress not only propels the application of translation technology and reshapes the translation industry but also profoundly impacts the realms of language learning and translation. This paper, situated against the backdrop of artificial intelligence and big data, focuses on the fundamental aspects of text stratification and corpus construction. Building upon discussions about text stratification and corpus construction, this paper extensively examines how human and machine interaction can effectively balance translation quality and cost, maximizing translation efficiency. Additionally, this paper proposes several innovation suggestions regarding future bilingual translation pedagogy.

The Machine Translation Post-Editing Annotation System (MTPEAS)

Abstract Machine translation post-editing quality evaluation has received relatively little attention in translation pedagogy to date. It is a time-consuming process that involves the comparison of three texts (source text, machine translation and student post-edited text) and the systematic identification and correction of students’ edits (or absence thereof) of machine translation (MT) output. There are as yet no widely available, standardized, user-friendly annotation systems for use in translator education. In this article, we address this gap by describing the Machine Translation Post-Editing Annotation System (MTPEAS). MTPEAS includes a taxonomy of seven categories that are presented in easy-to-understand terms: Value-adding edits, Successful edits, Unnecessary edits, Incomplete edits, Error-introducing edits, Unsuccessful edits, and Missing edits. We then assess the robustness of the MTPEAS taxonomy in a pilot study of 30 students’ post-edited texts and offer some preliminary findings on students’ MT error identification and correction skills.

Universal Basic Income as a Response to Automation? Attitudes of Human Translators Facing Neural Machine Translation

How do professionals who may be at risk of job loss due to AI-driven automation view a universal basic income (UBI) as a policy response to technological unemployment? This article examines the attitudes of translators from Turkey, a country with the highest perceived technological risks across the OECD. Based on interviews, the article reveals varying views on UBI, the framings of which reflect participants’ political views and understandings of capitalist societies. While some participants depict a perspective of economic individualism, emphasizing individual responsibility and a work ethic in opposing UBI, others advocate an anti-capitalist outlook, promoting collective action yet appearing skeptical of UBI. However, a majority supports UBI as a social right. Furthermore, those concerned about job or wage loss are primarily found among those who favor UBI and those who specialize as freelancers in areas perceived as more susceptible to automation, suggesting that job insecurity shapes views of UBI.

Artificial Intelligence (AI) Challenges and Opportunities in Translation : An African Experience

The Artificial Intelligence (AI) revolution has become a reality in today’s world and its importance for linguistics was recognized very early. Despite its unprecedent surge and integration into various academic fields including language teaching and translation, surprisingly, little work has been done by scholars in advancing discussions on the profound impact of the AI on the diversity of widely available languages in both developed and developing world. Africa is linguistically diverse continent with about one third of the world’s languages that are vastly underrepresented in the global digital data pool. AI translation machine is supported in only 25 languages out of over 2000 languages in the continent. The paper deploys homomorphism model of AI theory to interrogate the natural language data drawn the African languages to present the current and future challenges, opportunities and potential for developing AI algorithms that could fit neatly into the translation of the African languages. Most of the discussions in the paper focuses on the seven patterns of the AI, the usage and implementation of AI algorithms in the translation science. The research findings show some of the complexities of the African languages in which their syntactic categories have multiple corresponding semantic objects. Unlike English, the findings also reveal that syntactic operation in the African languages do not always have one corresponding semantic operation as postulated by the homomorphism model of AI theory. the study contributes to scholarly literature by stressing the limits and opportunities that relate to using AI in translation science and supplying input from NLP algorithms practitioners to expand the AI applicability operation in the translation science.

A Study on How Globalization Influences Translation Practices and the Demand for Translators in Various Industries

This study investigates the impact of globalization on translation practices and the demand for translators across various industries. Utilizing a mixed-methods research design, the study combines quantitative surveys and qualitative interviews with professional translators, translation agency managers, and industry experts. The findings reveal that technological advancements, particularly machine translation (MT) and computer-assisted translation (CAT) tools have significantly altered translation workflows, increasing efficiency while necessitating careful quality control. The demand for localization services has surged, especially in the tech industry, reflecting a broader trend towards culturally adapted content in global markets. Additionally, the study highlights a notable shift towards freelancing and remote work among translators, driven by the desire for flexibility but accompanied by challenges such as job stability and securing consistent work. Emerging challenges identified include maintaining translation quality amidst high workloads, adapting to rapidly evolving technologies, and navigating cultural differences to ensure culturally appropriate translations. Economic pressures, such as low rates and payment issues, further complicate the landscape for professional translators. These findings align with existing literature, emphasizing the need for continuous adaptation, professional development, and strategic approaches to address the dynamic and evolving nature of the translation industry in a globalized context. The study provides valuable insights into the current state of the translation industry, highlighting the importance of cultural competence, technological proficiency, and economic resilience for translators operating in a globalized world. Future research should explore the long-term impacts of these trends and the evolving role of translators in a rapidly changing global market.

Analysis of Dataset Limitations in Semantic Knowledge-Driven Multi-Variant Machine Translation

In this study, we explore the implications of dataset limitations in semantic knowledge-driven machine translation (MT) for intelligent virtual assistants (IVA). Our approach diverges from traditional single-best translation techniques, utilizing a multi-variant MT method that generates multiple valid translations per input sentence through a constrained beam search. This method extends beyond the typical constraints of specific verb ontologies, embedding within a broader semantic knowledge framework. We evaluate the performance of multi-variant MT models in translating training sets for Natural Language Understanding (NLU) models. These models are applied to semantically diverse datasets, including a detailed evaluation using the standard MultiATIS++ dataset. The results from this evaluation indicate that while multi-variant MT method is promising, its impact on improving intent classification (IC) accuracy is limited when applied to conventional datasets such as MultiATIS++. However, our findings underscore that the effectiveness of multi-variant translation is closely associated with the diversity and suitability of the datasets utilized. Finally, we provide an in-depth analysis focused on generating variant-aware NLU datasets. This analysis aims to offer guidance on enhancing NLU models through semantically rich and variant-sensitive datasets, maximizing the advantages of multi-variant MT.

Speech recognition and intelligent translation under multimodal human–computer interaction system

Abstract The traditional translation robot is limited to the translation of single-mode text images and text videos, which has the problem of low translation accuracy. Therefore, speech recognition and intelligent translation in multimodal human–computer interaction (HCI) system are proposed. First, the network structure of speech recognition model in multi-channel HCI system is established, and the multi-head self-attention mechanism is constructed. Then, the artificial intelligence voice wake-up function is designed, and a multimodal machine translation model is constructed. On this basis, selective attention is added to obtain visual recognition of perceived text, and the decoder is used for multimodal gating fusion to realize the output of encoder translation results. Experimental results show that this method has high BLUE value and high translation accuracy.

SPECIALIZED NEURAL MACHINE TRANSLATION FOR AVIATION ENGLISH: INSIGHTS FROM PORTUGUESE-TO-ENGLISH AVIATION SAFETY REPORTS

Aviation safety reports are essential sources for the identification and analysis of risks in civil aviation. These reports are written in plain language, which requires the application of Natural Language Processing techniques for automatic and intelligent treatment. In the case of Brazil, the vast majority of reports are written in Portuguese. Therefore, for comparison with international database of reports that are written in English, a first step is the translation of Brazilian reports. In this work, a proposal for a machine translation model is presented based on the fine-tuning of pre-trained models. To this end, an aviation-specific language corpus is developed with the objective of generating example data for model training. Finally, a pre-trained model is fine-tuned with the corpus created in order to implement an automatic translation model that achieves good results in the task considering the specifics of the aviation language. As a result, a first model is implemented, presenting coherent results of translation between Portuguese/English in the specific domain of aviation.

Human versus Neural Machine Translation Creativity: A Study on Manipulated MWEs in Literature

In the digital era, the (r)evolution of neural machine translation (NMT) has reshaped both the market and translators’ workflow. However, the adoption of this technology has not fully reached the creative field of literary translation. Against this background, this study aims to explore to what extent NMT systems can be used to translate the creative challenges posed by idioms, specifically manipulated multiword expressions (MWEs) found in literary texts. To carry out this pilot study, five manipulated MWEs were selected from a fantasy novel and machine-translated (English > Spanish) by four NMT systems (DeepL, Google Translate, Bing Translator, and Reverso). Then, each NMT output as well as a human translation are assessed by six professional literary translators by using a human evaluation sheet. Based on these results, the creativity obtained in each translation method was calculated. Despite the satisfactory performance of both DeepL and Google Translate, HT creativity was highly superior in almost all manipulated MWEs. To the best of our knowledge, this paper not only contributes to the ongoing study of NMT applied to literature, but it is also one of the few studies that delve into the almost unexplored field of assessing creativity in neural machine-translated MWEs.

Indirect (pivot) audiovisual translation: a conversation with and among B. Agulló Garcia, D. Orrego-Carmona and G. Spiteri Miggiani

ABSTRACT Through a series of questions and responses, in this dialogue-based article we aim to stimulate reflection on indirect translation within the audiovisual translation community, offering perspectives on this practice. We start by delving into the historical roots of indirect translation in the industry, questioning how far back such practices extend. Then we shift the focus to the challenges translators face in indirect translation workflows, to the potential contributions of research to the ongoing debates surrounding indirect translation, as well as to how indirect translation and machine translation intersect. Finally, looking to the future, we examine potential developments in indirect translation workflows and consider how educational programs can adapt to meet the changing demands of the industry.