Reference Translations Research Articles

« Vous pouvez entendre les mâchoires de l’arme de l’ange ». Le traducteur humain face à la traduction automatique dans le contexte littéraire : Les Livres de Jakób d’Olga Tokarczuk

The dynamic development of automatic translators raises questions about the future of human translators. The aim of this article is to evaluate the French versions of an excerpt from The Books of Jacob by Olga Tokarczuk, made by Systran, Google Translate and DeepL, and to compare them with the reference translation made by a human translator. The translations were carried out in 2019 as part of the ATLAS association project ‘L’Observatoire de la traduction automatique’. The analyses show that the inability to interpret the original in a complex way leads to weaknesses in machine translators. This proves that the position of the human translator remains unthreatened in the field of translating high-quality literary texts.

An Intelligent Image Caption Generator Model using Deep Learning

Replicating the innate human capability to understand image content and create descriptive text is a formidable challenge for machines. The application of image captioning is highly versatile and holds great importance. It involves the creation of concise and descriptive captions for images using a variety of advanced techniques such as Natural Language Processing (NLP), Computer Vision (CV), and Deep Learning (DL). These techniques enable the automated generation of accurate and meaningful captions, which have numerous practical applications across industries such as healthcare, entertainment, marketing to mention but few. This study employed the machine learning algorithms of Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) to train computer vision using the Hibs dataset. The model leveraged input images with associated captions to identify and analyze predicted captions within the computer system. Model evaluation is a step in assessing the model's performance. This involves metrics such as BLEU score and METEOR score to measure the quality of the model's output in comparison to reference translation. These metrics help in quantifying the accuracy and fluency of the generated text, and the results are then presented to provide insights into the model's performance.

Evaluating optimal reference translations

Abstract The overall translation quality reached by current machine translation (MT) systems for high-resourced language pairs is remarkably good. Standard methods of evaluation are not suitable nor intended to uncover the many translation errors and quality deficiencies that still persist. Furthermore, the quality of standard reference translations is commonly questioned and comparable quality levels have been reached by MT alone in several language pairs. Navigating further research in these high-resource settings is thus difficult. In this paper, we propose a methodology for creating more reliable document-level human reference translations, called “optimal reference translations,” with the simple aim to raise the bar of what should be deemed “human translation quality.” We evaluate the obtained document-level optimal reference translations in comparison with “standard” ones, confirming a significant quality increase and also documenting the relationship between evaluation and translation editing.

Comparative assessment of Bing Translator and Youdao Machine Translation Systems in English-to-Chinese literary text translation

This study explores the performance of machine translation of literary texts from English to Chinese. The study compares two machine translation systems, Bing Translator and Youdao Machine Translation, using selected texts from the novel “Nineteen eighty-four” by George Orwell. The data collection includes the original source texts, their machine-generated translations by Bing Translator and Youdao Machine Translation, and comparisons with human reference translations to assess the performance of these systems. The research’s focal point is to evaluate the accuracy, fluency, and appropriateness of translations generated by these two machine translation systems, while also analyzing the post-editing effort required to enhance the quality of the final machine-translated product. The study revealed that despite the presence of flaws in both machine translation systems, Youdao Machine Translation demonstrated superior performance, especially in accurately translating technical terms and idiomatic expressions, making it the more effective option overall. Nevertheless, the translations from Youdao Machine Translation required more substantial post-editing efforts to improve fluency and readability. Conversely, Bing Translator yielded more fluent and natural-sounding translations, albeit with a need for improved accuracy in translating technical terms and idiomatic expressions. The study concludes that while machine translation systems are capable of generating reasonable translations for literary texts, human post-editing remains essential to ensure the final output’s accuracy, fluency, and appropriateness. The study underscores the importance of selecting the appropriate machine translation system based on the nature of the text being translated. It also highlights the critical role of post-editing in refining the quality of machine-translated outputs, suggesting that while machine translation can provide a solid foundation, human intervention is indispensable for achieving optimal accuracy, fluency, and overall readability in literary translations.

Culturally based semantic losses in Lonely Planet’s travel guides translations for Beijing, Shanghai, and Sichuan

Translating cultural references in tourism materials is pivotal in bridging cultural gaps and facilitating cross-cultural communication. Despite the escalating demand for accurate translation, no research exists to address the losses in cultural connotations and their impact on semantic accuracy in Chinese-English cultural reference translation within Lonely Planet’s travel guides. This research seeks to fill this gap, specifically focusing on Beijing, Shanghai, and Sichuan destinations. The objectives are as follows: 1) to identify the types of cultural connotation losses in the English translation of Chinese cultural references; 2) to examine cultural-related semantic losses, considering instances where cultural connotation losses lead to partial or complete semantic losses; and 3) to elucidate the translation decisions (both macro and micro levels) implications on the culturally based semantic losses. A qualitative-descriptive approach forms the foundation of this research. The findings revealed seven types of cultural connotation losses, with partial semantic losses predominant. Applying Venuti’s domestication and foreignization, the results also uncovered a strong inclination toward foreignization, emphasizing the strangeness inherent in the source culture is intensified. These findings contribute to a nuanced understanding of the complexities of translating cultural connotations and maintaining semantic accuracy, offering a comprehensive typology that can guide future translation practices and serve as a springboard for further research in the field. Lastly, this study underscores the significance of maintaining cultural connotations in translation, thereby contributing to the ongoing development of cross-cultural communication.

A comparative analysis of lexical-based automatic evaluation metrics for different Indic language pairs

<p>With the rise of machine translation systems, it has become essential to evaluate the quality of translations produced by these systems. However, the existing evaluation metrics designed for English and other European languages may not always be suitable or apply to other Indic languages due to their complex morphology and syntax. Machine translation evaluation (MTE) is a process of assessing the quality and accuracy of the machine-translated text. MTE involves comparing the machine-translated output with the reference translation to calculate the level of similarity and correctness. Therefore, this study evaluates different metrics, namely, BLEU, METEOR, and TER to identify the most suitable evaluation metric for Indic languages. The study uses datasets for Indic languages and evaluates the metrics on various translation systems. The study contributes to the field of MT by providing insights into suitable evaluation metrics for Indic languages. This research paper aims to study and compare several lexical automatic machine translation evaluation metrics for Indic languages. For this research analysis, we have selected five language pairs of parallel corpora from the low-resource domain, such as English–Hindi, English-Punjabi, English-Gujarati, English-Marathi, and English-Bengali. All these languages belong to the Indo-Aryan language family and are resource-poor. A comparison of the state of art MT is presented and shows which translator works better on these language pairs. For this research work, the natural language toolkit tokenizers are used to assess the analysis of the experimental results. These results have been performed by taking two different datasets for all these language pairs using fully automatic MT evaluation metrics. The research study explores the effectiveness of these metrics in assessing the quality of machine translations between various Indic languages. Additionally, this dataset and analysis will make it easier to do future research in Indian MT evaluation.</p>

Method for three-dimensional reconstruction of dynamic stereo vision based on line structured light using global optimization

To overcome the limitation of the measurement range of static stereo vision, a three-dimensional (3-D) reconstruction method of dynamic stereo vision based on line structured light for metal surface is presented in this paper. The dynamic stereo vision is mainly composed of binocular cameras and a laser line generator, and moves along a horizontal guide rail. The intrinsic parameters of the cameras and the rotation and translation of the right camera relative to the left camera are previously obtained by Zhang’s method. To obtain the 3-D coordinates of the line structured light fringe in each position, the sub-pixel coordinates of the fringe center are extracted by Steger algorithm, and a joint calibration method between single camera and a laser line generator is proposed. The joint calibration method calculates the plane equation of the line structured light by using multiple checkerboard images with projected structured light line. To unify the 3-D coordinates of structured light fringe in each position to the reference coordinate system, the rotation and translation of the camera in each position relative to the reference position (called absolute pose) should be solved. The direct linear transformation method is firstly used to calculate the rough relative pose between adjacent frames, and principle of re-projection error minimization in object space is used to obtain the refined relative pose. Instead of using step-by-step transfer method, the absolute pose of the camera in each position is eventually calculated by using global optimization algorithm after the relative pose between all adjacent frames are obtained. Computer simulation and abundant experimental data validate the effectiveness of proposed method. The proposed 3-D reconstruction method using global optimization is respectively compared with step-by-step transfer method based on pixel re-projection error minimization and re-projection error minimization in object space. The experimental results reflect that our method is superior to step-by-step transfer method, and the 3-D reconstruction accuracy is improved by 28.05 % after global optimization.

The Negative Transfer Effect on the Neural Machine Translation of Egyptian Arabic Adjuncts into English: The Case of Google Translate

Parallel corpora for low-resource Arabic dialects and English are limited and small-scale, and most neural machine translation models, including Google Translate, rely mainly on parallel corpora of standard Arabic and English to train for dialectal Arabic translation. A model well trained to translate to and from standard Arabic is believed to efficiently translate dialectal Arabic, given their similarities. This study demonstrates the impact of not using large-scale, dialect-specific parallel corpora by quantitatively and qualitatively analyzing the performance of Google Translate in translating Egyptian Arabic adjuncts. Compared to human reference translation, Google Translate achieved a low BLEU score of 14.69. Qualitative analysis showed that reliance on standard Arabic parallel corpora caused a negative transfer problem manifested in the literal translation of idiomatic adjuncts, the misinterpretation of dialectal adjuncts as main clause constituents, the translation of dialectal adjuncts after orthographically similar standard Arabic words, and the use of standard Arabic common lexical meanings to translate dialect-specific adjuncts. This study’s findings will be relevant for researchers interested in dialectal Arabic neural machine translation and has implications for investment in the development of large-scale, dialect-specific corpora to better process the peculiarities of Arabic dialects and reduce the effect of negative transfer from standard Arabic.

Denoising Pre-training for Machine Translation Quality Estimation with Curriculum Learning

Quality estimation (QE) aims to assess the quality of machine translations when reference translations are unavailable. QE plays a crucial role in many real-world applications of machine translation. Because labeled QE data are usually limited in scale, recent research, such as DirectQE, pre-trains QE models with pseudo QE data and obtains remarkable performance. However, there tends to be inevitable noise in the pseudo data, hindering models from learning QE accurately. Our study shows that the noise mainly comes from the differences between pseudo and real translation outputs. To handle this problem, we propose CLQE, a denoising pre-training framework for QE based on curriculum learning. More specifically, we propose to measure the degree of noise in the pseudo QE data with some metrics based on statistical or distributional features. With the guidance of these metrics, CLQE gradually pre-trains the QE model using data from cleaner to noisier. Experiments on various benchmarks reveal that CLQE outperforms DirectQE and other strong baselines. We also show that with our framework, pre-training converges faster than directly using the pseudo data. We make our CLQE code available (https://github.com/NJUNLP/njuqe).

A case study on the evaluation of Korean-English legal translations by generic and custom neural machine translation engines

This paper addresses the quality of Korean-English legal translation outputs generated by three neural machine translation engines, namely NMT1, NMT2 and NMT3. NMT1 and NMT2 are widely available multilingual generic machine translation engines, whereas NMT3 is a custom machine translation engine for legal translation. In both automatic and human evaluations of the three engines’ English translations of Korean statutes, the custom engine outperformed the other two NMTs. BLEU and METEOR scores revealed that NMT3’s output was more similar to human reference translation. Human evaluation results confirmed NMT3’s strong performance, outperforming the generic engines in accuracy and fluency while underperforming in terminology. However, further analysis revealed that NMT3 was superior in handling terms comprising compound nouns and sentences over 25 words. Although the custom engine’s performance still requires substantial human involvement for quality translation, it showed a potential for the legal domain, demonstrating human parity in short sentences. The growing demand for speedy translation under budget constraints calls for further research on technology-based translation tools and human-computer interaction for legal translation.

Metamorphic Robustness Testing for DeepL Translation

Due to the complexity and flexibility of natural languages, such robustness testing involving the ability of machine translation software to handle incorrect or unintended input are extremely challenging, especially if there is no human oracle or reference translation. In this paper, using metamorphic robustness testing methods, we compare the translation of the primitive input with the translation of a subsequent input that has different types of very small changes and may cause large changes in the translation results. The lack of robustness of the DeepL translation is shown by our experiments, these preliminary results show that metamorphic testing has potential in the field of natural language processing.

A Survey on Evaluation Metrics for Machine Translation

The success of Transformer architecture has seen increased interest in machine translation (MT). The translation quality of neural network-based MT transcends that of translations derived using statistical methods. This growth in MT research has entailed the development of accurate automatic evaluation metrics that allow us to track the performance of MT. However, automatically evaluating and comparing MT systems is a challenging task. Several studies have shown that traditional metrics (e.g., BLEU, TER) show poor performance in capturing semantic similarity between MT outputs and human reference translations. To date, to improve performance, various evaluation metrics have been proposed using the Transformer architecture. However, a systematic and comprehensive literature review on these metrics is still missing. Therefore, it is necessary to survey the existing automatic evaluation metrics of MT to enable both established and new researchers to quickly understand the trend of MT evaluation over the past few years. In this survey, we present the trend of automatic evaluation metrics. To better understand the developments in the field, we provide the taxonomy of the automatic evaluation metrics. Then, we explain the key contributions and shortcomings of the metrics. In addition, we select the representative metrics from the taxonomy, and conduct experiments to analyze related problems. Finally, we discuss the limitation of the current automatic metric studies through the experimentation and our suggestions for further research to improve the automatic evaluation metrics.

Stylistic Anomalies in Some Arabic Texts with Reference Translation into English

It is quite essential, for whose who are highly interested in translation and stylistic studies, to lay great focus on the main issues which are closely related to both translation and stylistic anomalies. It is to be noted that stylistic anomalies can create aesthetic values to any text which implies them. It is also far better to take the phenomenon of anomaly into consideration because it is not touched upon before. As a matter of fact, the phenomenon of anomalies poses a great challenge to the translators of the Arabic texts since they neither can be true nor false. Thus, enough attention should be paid to the slippery areas of anomalies in some Arabic texts from stylistic and translational perspective.

A Study of Formal Features of Chu Ci Translation and Research Texts in English Based on Variational Translation Theory

The dissemination works of Chu Ci (楚辞) in Western countries can roughly be divided into three groups: translation works, research works and literary works. The formal features of the dissemination works are intimately related to and determined by the variational translation methods which include adaptation translation, selective translation, translation and comment, translation and writing, introduction-translation, edited translation, reference translation, condensed translation, and so on. Research on the textual features of Chu Ci dissemination works in the West is of great help to the globalization of Chinese culture, especially to ancient Chinese literary and cultural classics’ going global which is an important national policy of the Chinese government.

Leveraging machine translation for cross-lingual fine-grained cyberbullying classification amongst pre-adolescents

AbstractCyberbullying is the wilful and repeated infliction of harm on an individual using the Internet and digital technologies. Similar to face-to-face bullying, cyberbullying can be captured formally using the Routine Activities Model (RAM) whereby the potential victim and bully are brought into proximity of one another via the interaction on online social networking (OSN) platforms. Although the impact of the COVID-19 (SARS-CoV-2) restrictions on the online presence of minors has yet to be fully grasped, studies have reported that 44% of pre-adolescents have encountered more cyberbullying incidents during the COVID-19 lockdown. Transparency reports shared by OSN companies indicate an increased take-downs of cyberbullying-related comments, posts or content by artificially intelligen moderation tools. However, in order to efficiently and effectively detect or identify whether a social media post or comment qualifies as cyberbullying, there are a number factors based on the RAM, which must be taken into account, which includes the identification of cyberbullying roles and forms. This demands the acquisition of large amounts of fine-grained annotated data which is costly and ethically challenging to produce. In addition where fine-grained datasets do exist they may be unavailable in the target language. Manual translation is costly and expensive, however, state-of-the-art neural machine translation offers a workaround. This study presents a first of its kind experiment in leveraging machine translation to automatically translate a unique pre-adolescent cyberbullying gold standard dataset in Italian with fine-grained annotations into English for training and testing a native binary classifier for pre-adolescent cyberbullying. In addition to contributing high-quality English reference translation of the source gold standard, our experiments indicate that the performance of our target binary classifier when trained on machine-translated English output is on par with the source (Italian) classifier.

Machine vs human translation: a new reality or a threat to professional Arabic–English translators

PurposeHow closely does the translation match the meaning of the reference has always been a key aspect of any machine translation (MT) service. Therefore, the primary goal of this research is to assess and compare translation adequacy in machine vs human translation (HT) from Arabic to English. The study looks into whether the MT product is adequate and more reliable than the HT. It also seeks to determine whether MT poses a real threat to professional Arabic–English translators.Design/methodology/approachSix different texts were chosen and translated from Arabic to English by two nonexpert undergraduate translation students as well as MT services, including Google Translate and Babylon Translation. The first system is free, whereas the second system is a fee-based service. Additionally, two expert translators developed a reference translation (RT) against which human and machine translations were compared and analyzed. Furthermore, the Sketch Engine software was utilized to examine the translations to determine if there is a significant difference between human and machine translations against the RT.FindingsThe findings indicated that when compared to the RT, there was no statistically significant difference between human and machine translations and that MTs were adequate translations. The human–machine relationship is mutually beneficial. However, MT will never be able to completely automated; rather, it will benefit rather than endanger humans. A translator who knows how to use MT will have an opportunity over those who are unfamiliar with the most up-to-date translation technology. As MTs improve, human translators may no longer be accurate translators, but rather editors and editing materials previously translated by machines.Practical implicationsThe findings of this study provide valuable and practical implications for research in the field of MTs and for anyone interested in conducting MT research.Originality/valueIn general, this study is significant as it is a serious attempt at getting a better understanding of the efficiency of MT vs HT in translating the Arabic–English texts, and it will be beneficial for translators, students, educators as well as scholars in the field of translation.

NMT verb rendering: A cognitive approach to informing Arabic-into-English post-editing

Abstract Machine translation (MT) has made significant strides and has reached accuracy levels that often make the post-editing (PE) of MT output a viable alternative to manual translation. However, despite professional translators increasingly considering PE as a valid stage in their translation workflow, little has been done to investigate MT output for the purpose of informing training in PE. Against this background, the present project focuses on the handling of tense and aspect configurations in the English translation of Arabic sentences using current neural machine translation (NMT) systems. Using a dataset of representative Arabic sentences, the output of five NMT engines was assessed against reference translations. The investigation reveals regressing accuracy levels when comparing morphological, structural, and contextual tenses. These findings are believed to represent valuable information that contributes to a more informed training in the PE of Arabic-into-English NMT output.

Lexical Diversity in Statistical and Neural Machine Translation

Neural machine translation systems have revolutionized translation processes in terms of quantity and speed in recent years, and they have even been claimed to achieve human parity. However, the quality of their output has also raised serious doubts and concerns, such as loss in lexical variation, evidence of “machine translationese”, and its effect on post-editing, which results in “post-editese”. In this study, we analyze the outputs of three English to Slovenian machine translation systems in terms of lexical diversity in three different genres. Using both quantitative and qualitative methods, we analyze one statistical and two neural systems, and we compare them to a human reference translation. Our quantitative analyses based on lexical diversity metrics show diverging results; however, translation systems, particularly neural ones, mostly exhibit larger lexical diversity than their human counterparts. Nevertheless, a qualitative method shows that these quantitative results are not always a reliable tool to assess true lexical diversity and that a lot of lexical “creativity”, especially by neural translation systems, is often unreliable, inconsistent, and misguided.

Japanese Translation Quality Evaluation System Based on Deep Neural Network Algorithm

The current translation quality evaluation system relies on the combination of manual and text comparison for evaluation, which has the defects of low efficiency and large evaluation errors. In order to optimize the defects of the current quality evaluation system, a Japanese translation quality evaluation system based on deep neural network algorithm will be designed. In order to improve the processing efficiency of the system, the USB3.0 communication module of the hardware system will be optimized. Based on the hardware design, the reference translation map is used to extend the reference translation of Japanese translation. The evaluation indexes of over- and under-translation are set, and the evaluation of Japanese translation quality is realized after the parameters are determined by training the deep neural network using the sample set. The system functional test results show that the average data transmission processing time of the system is improved by about 31.27%, and the evaluation error interval is smaller and the evaluation is more reliable.

Embracing policy implementation science to ensure translation of evidence to cancer control policy.

Although health-related policies are abundant, efforts to understand how to ensure that these policies serve as an effective vehicle for translating scientific evidence are relatively sparse. This paper explores how policy-focused implementation science (IS) may contribute to understanding the translation of scientific evidence to health-related policy in governmental and nongovernmental sectors. Expanding the focus of implementation science in cancer control could systematically address policy to both increase the use of scientific evidence in general and to address health equity. In this Commentary, we look to relevant work outside of IS that could be informative, most notably from the field of political science. We propose several ideas for future research that could help move the field of policy implementation science in cancer control in the USA forward. Although most efforts to increase uptake of the scientific evidence base reference translation to "practice and policy," there has been relatively little emphasis in the USA on implementation at the policy level, especially related to cancer control. If we are to achieve the full benefits of scientific discovery on population and public health, we will need to consider policy as a critical mechanism by which evidence can be translated to practice. We have a robust set of methods within implementation science that are increasing the pace of adoption and maintenance of evidence-based programs in a variety of settings. Building on these efforts, the time is right to expand our focus to include policy implementation.