Machine Translation Quality Research Articles

Challenges of machine translation application to teaching ESP to construction students

The article deals with problems of translation teaching, namely machine translation for ESP students. The study aims to conduct a comparative analysis of machine and human translation of construction terminology, identify causes of errors, provide recommendations for improving quality of students’ translation via post-editing as well as developing their interdisciplinary skills using CAT tools. The main research methods include comparative and contrastive analysis as well as the quantitative method. The research material is presented by tittles of construction students’ qualification papers translated from Ukrainian into English. Quality of machine translation is affected by peculiarities of construction terminology due to harmonization of terms according to Eurocodes. The authors prove that application of software to translating construction texts without further proofreading or post-editing by students themselves entails errors including distortion of terminology and, consequently, meaning. There are presented reasons for lexical errors caused by peculiarities of translating multicomponent terms, discrepancies in translating prepositions. The article is intended for a wide range of specialists interested in translating construction texts and teaching ESP. Based on the results obtained, the authors develop recommendations for translating construction texts by using machine translation accompanied by students’ post-editing.

Morphology & word sense disambiguation embedded multimodal neural machine translation system between Sanskrit and Malayalam

Machine translation is the process of conversion from one language to another using computers. In spite of the existence of several traditional machine translation systems like rule-based, statistical etc., the development of neural machine translation has significantly improved translation quality. However, the morphological richness and agglutinative nature of Indian languages hampers the usage of neural machine translation for translation among Indian languages despite its potential to overcome the difficulties posed by conventional translation systems. Hence the combination of neural machine translation with morphology, part of speech tagger and word sense disambiguation is proposed to address these challenges and adapt it for bidirectional translation between Sanskrit and Malayalam. Conventional neural machine translations are unimodal systems that utilize textual data for translation. Translation quality of conventional neural machine translation is enhanced with a novel multimodal bidirectional neural machine translation system with text & speech as the two modalities based on parallel text corpus and speech database. The best performance score is obtained for two level fusion, the feature vectors extracted from speech signals using different transforms (Wavelet transform, 1-D sequency mapped real transform & 1-D GCD based mapped real transform) are fused and which is further fused with a context vector from the text modality. Automatic and manual review methods are employed to assess quality of translation. Findings from the experiments reveal that addition of speech modality improved the overall quality of translation. BLEU scores of 43.89 and 42.72 are obtained for Sanskrit to Malayalam and Malayalam to Sanskrit multimodal translation respectively.

Reduction of Neural Machine Translation Failures by Incorporating Statistical Machine Translation

This paper proposes a hybrid machine translation (HMT) system that improves the quality of neural machine translation (NMT) by incorporating statistical machine translation (SMT). Therefore, two NMT systems and two SMT systems were built for the Slovenian–English language pair, each for translation in one direction. We used a multilingual language model to embed the source sentence and translations into the same vector space. From each vector, we extracted features based on the distances and similarities calculated between the source sentence and the NMT translation, and between the source sentence and the SMT translation. To select the best possible translation, we used several well-known classifiers to predict which translation system generated a better translation of the source sentence. The proposed method of combining SMT and NMT in the hybrid system is novel. Our framework is language-independent and can be applied to other languages supported by the multilingual language model. Our experiment involved empirical applications. We compared the performance of the classifiers, and the results demonstrate that our proposed HMT system achieved notable improvements in the BLEU score, with an increase of 1.5 points and 10.9 points for both translation directions, respectively.

원천텍스트 난이도와 기계번역 품질에 대한 고찰 - 구글 한영 법령번역 사례를 중심으로

This paper attempts to investigate the relationship between the source text difficulty and the quality of machine translation output. We examined the results of human evaluation of the Korean-English statutory translation by Google Translate and the source text difficulty measured in terms of lexical and terminological difficulty and syntactic complexity. The translations of two different statutes revealed varying correlation patterns examined in terms of fluency, accuracy, terminology and others. The results demonstrate that the two source texts’ syntactic complexity had a negative correlation with the machine translation quality evaluation. In other words, Google Translate was found to be weak at translating long and syntactically complex sentences. On the other hand, lexical and terminological difficulty did not have any correlation with the results of human evaluation despite the fact that statutes contain a lot of difficult words. Overall, the source text difficulty had a strong negative correlation with the human evaluation results, which was generally supported by the evaluators’ comments on fluency and accuracy errors.

기계 번역과 자연어처리를 활용한 튜터링 활동 분석

This is a follow-up study by Yoon, Kim, Choi, and Kim (2021). In the current study, the same text as the previous study was machine-translated into English, and then the topics derived through a statistical model, which were compared with the categories derived from the Korean text of the previous study. By doing this, rather than simply evaluating the quality of machine translation, the present study examined 1) the natural language processing model applied to text, 2) the role of human researchers in the process of interpreting the results after fitting the model, and 3) the harmony between human and machine translation, The texts to be analyzed were collected from the liberal arts education tutoring program of D university. A theoretical description of the Latent Dirichlet Allocation model, which is often used for topic modeling, and an explanation of the use of the R library ‘topicmodels’ were added. The analyses revealed a somewhat different pattern of topics from the previous study, and the authors’ interpretation was presented. Finally, while acknowledging the explosive growth of natural language processing and machine translation as a branch of it that cannot be reversed, the room for human translators to complement machine translation was suggested. This study presents a new approach to liberal arts education research by combining data analysis techniques using natural language processing and machine translation discussions.

TECHNOLOGY FOR GRAMMATICAL ERRORS CORRECTION IN UKRAINIAN TEXT CONTENT BASED ON MACHINE LEARNING METHODS

Context. Most research in grammatical and stylistic error correction focuses on error correction in English-language textual content. Thanks to the availability of large data sets, a significant increase in the accuracy of English grammar correction has been achieved. Unfortunately, there are few studies on other languages. Systems for the English language are constantly developing and currently actively use machine learning methods: classification (sequence tagging) and machine translation. A large amount of parallel or manually labelled data is required to build a high-quality machine learning model for correcting grammatical/stylistic errors in the texts of those morphologically complex languages. Manual data annotation requires a lot of effort by professional linguists, which makes the creation of text corpora, especially in morphologically rich languages, mainly Ukrainian, a time- and resource-consuming process.
 Objective of the study is to develop a technology for correcting errors in Ukrainian-language texts based on machine learning methods using a small set of annotated parallel data.
 Method. For this study, machine learning algorithms were selected when developing a system for correcting errors in Ukrainianlanguage texts using an optimal pipeline, including pre-processing and selecting text content and generating features in small annotated data corpora. The neural network’s use with a new architecture, a review of state-of-the-art methods, and a comparison of different pipeline stages will make it possible to determine such a combination of them, allowing a high-quality error correction model in Ukrainian-language texts.
 Results. A machine learning model for error correction in Ukrainian-language texts has been developed. A universal scheme for creating an error correction system for different languages is proposed. According to the results, the neural network can correct simple sentences written in Ukrainian. However, creating a full-fledged system will require spell-checking using dictionaries and checking rules, both simple and based on the result of parsing dependencies or other features. The pre-trained neural translation model mT5 has the best performance among the three models. To save computing resources, it is also possible to use a pre-trained BERT-type neural network as an encoder and a decoder. Such a neural network has half the number of parameters as other pretrained machine translation models and shows satisfactory results in correcting grammatical and stylistic errors.
 Conclusions. The created model shows excellent classification results on test data. The calculated machine translation quality metrics allow only a partial comparison of the models since most of the words and phrases in the original and corrected sentences are the same. The best value for both BLEU (0.908) and METEOR (0.956) is obtained for mT5, which is consistent with the case study in which the most accurate error corrections without changing the initial value of the sentence are obtained for such a neural network. The M2M100 has a higher BLEU score (0.847) than the “Ukrainian Roberta” Encoder-Decoder (0.697). However, subjectively evaluating the results of the correction of examples, the M2M100 does a much worse job than the other two models. For METEOR, M2M100 (0.925) also has a higher score than the “Ukrainian Roberta” Encoder-Decoder (0.876).

Saudi Translation Agencies and Translation Centers: A Study of Post-Editing Practices

This qualitative research aims to investigate and analyze the practice of post-editing activity in Saudi translation agencies and translation centers. The study’s focus is on the importance of post-editing Machine Translation (MT) output as an approach to upgrade the quality of MT results to enhance the accuracy of translations. The study data were collected through an electronic survey (questionnaire) comprising short questions about post-editing practice and guidelines in addition to their knowledge about post-editing certification programs. The participants in this study were 18 professional translators working in certified translation agencies and translation centers across different universities between Jeddah, Dammam, and Riyadh cities. The results of the study showed that the participants post-edited the MT outputs in their tasks and recognized the importance of being aware of post-editing standards and guidelines. Further, the findings demonstrated that the majority of the participants used machine translation tools. Finally, they were positively disposed about post-editing certification programs to take employment as post-editors. Based on the results of this study, it is recommended to analyze the state of post-editing in Saudi Arabia on a larger sample to investigate the attitudes of Saudi translation students towards practicing post-editing of MT outputs, and analyzing the activity of post-editing MT in specialized texts by Saudi professional translators.

Google Translate Errors in Legal Texts: Machine Translation Quality Assessment

Machine translation received intensive research in different language pairs; yet, the quality of specialized translation; e.g., legal translation, from Arabic into English received little attention. The present study investigates the quality of machine translation of legal texts from Arabic into English. The paper aims at examining the errors found in the machine translation of legal texts from Arabic into English. It also studies the legal discourse features in machine-translation output. The research questions tackle the accuracy of machine translation of legal discourse and error categories and frequencies in machine translation. The researcher evaluated several factors to assess the quality of Google Translate; i.e., lexical, syntactic, and register-related errors. The study data consists of five legislative texts. The researcher conducted a manual error assessment and classification. To ensure the reliability of the error analysis, an existing human translation of the documents was used as a reference to ensure the reliability of the MT quality assessment and post-editing process. Later, the errors were classified, and their percentages and frequencies were calculated. A few examples of errors in each category were discussed and analysed. The highest error category was lexical errors scoring 43.4%. The last detected error category was deletion with a percentage of 1.7%. Syntactic errors constituted one-fourth of the errors found in the data. Legal register-related errors scored 30.2%. The subcategories of legal register-related errors varied in their occurrence; e.g., one-third of the errors related to legal discourse were legal terms. The study concluded that machine translation; though it provided a comprehensible output, could not translate legal structures and terminology perfectly.

A Survey on Evaluation Metrics for Machine Translation

The success of Transformer architecture has seen increased interest in machine translation (MT). The translation quality of neural network-based MT transcends that of translations derived using statistical methods. This growth in MT research has entailed the development of accurate automatic evaluation metrics that allow us to track the performance of MT. However, automatically evaluating and comparing MT systems is a challenging task. Several studies have shown that traditional metrics (e.g., BLEU, TER) show poor performance in capturing semantic similarity between MT outputs and human reference translations. To date, to improve performance, various evaluation metrics have been proposed using the Transformer architecture. However, a systematic and comprehensive literature review on these metrics is still missing. Therefore, it is necessary to survey the existing automatic evaluation metrics of MT to enable both established and new researchers to quickly understand the trend of MT evaluation over the past few years. In this survey, we present the trend of automatic evaluation metrics. To better understand the developments in the field, we provide the taxonomy of the automatic evaluation metrics. Then, we explain the key contributions and shortcomings of the metrics. In addition, we select the representative metrics from the taxonomy, and conduct experiments to analyze related problems. Finally, we discuss the limitation of the current automatic metric studies through the experimentation and our suggestions for further research to improve the automatic evaluation metrics.

Is machine translation a dim technology for its users? An eye tracking study.

State-of-the-art research shows that the impact of language technologies on public awareness and attitudes toward using machine translation has been changing. As machine translation acceptability is considered to be a multilayered concept, this paper employs criteria of usability, satisfaction and quality as components of acceptability measurement. The study seeks to determine whether there are any differences in the machine-translation acceptability between professional users, i.e., translators and language editors, and non-professional users, i.e., ordinary users of machine translation who use it for non-professional everyday purposes. The main research questions whether non-professional users process raw machine translation output in the same way as professional users and whether there is a difference in the processing of raw machine-translated output between users with different levels of machine-translated text acceptability are analyzed. The results of an eye tracking experiment, measuring fixation time, dwell time and glance count, indicate a difference between professional and non-professional users' cognitive processing and acceptability of machine translation output: translators and language editors spend more time overall reading the machine-translated texts, possibly because of their deeper critical awareness as well as professional attitude toward the text. In terms of acceptability overall, professional translators critically assess machine translation on all components of which confirms the findings of previous similar research. However, the study draws attention to non-professional users' lower awareness regarding machine translation quality. The study was conducted within a research project that received funding from the Research Council of Lithuania (LMTLT, agreement No S-MOD-21-2), seeking to explore and evaluate the impact on society of machine translation technological solutions.

Textual Augmentation Techniques Applied to Low Resource Machine Translation: Case of Swahili

In this work we investigate the impact of applying textual data augmentation tasks to low resource machine translation. There has been recent interest in investigating approaches for training systems for languages with limited resources and one popular approach is the use of data augmentation techniques. Data augmentation aims to increase the quantity of data that is available to train the system. In machine translation, majority of the language pairs around the world are considered low resource because they have little parallel data available and the quality of neural machine translation (NMT) systems depend a lot on the availability of sizable parallel corpora. We study and apply three simple data augmentation techniques popularly used in text classification tasks; synonym replacement, random insertion and contextual data augmentation and compare their performance with baseline neural machine translation for English-Swahili (En-Sw) datasets. We also present results in BLEU, ChrF and Meteor scores. Overall, the contextual data augmentation technique shows some improvements both in the EN -> SW and SW -> EN directions. We see that there is potential to use these methods in neural machine translation when more extensive experiments are done with diverse datasets.

The Impact of Pre-Editing on the Quality of Machine Translation

The Impact of Pre-Editing on the Quality of Machine Translation

English Machine translation System Based on Neural Network Algorithm

In order to solve the problems of machine translation efficiency and translation quality, this paper proposes a research method of machine English translation evaluation system based on BP neural network algorithm. This method provides users with a more intelligent machine translation service experience. With the help of BP neural network algorithm, taking English online translation as the research object, Google's translation quality is the best, with an error frequency of only 167, while Baidu translation and iFLYTEK translation in China have a high error rate of 266 and 301 respectively, which is much higher than Google translation. A model of machine translation evaluation system based on neural network algorithm is proposed to better solve the disadvantages of traditional English machine translation. The results show that the machine translation system based on neural network algorithm can further optimize the problems existing in machine translation, such as insufficient use of information and large scale of model parameters, and further improve the performance of neural network machine translation.

Linearization Weight Compression and In-Situ Hardware-Based Decompression for Attention-Based Neural Machine Translation

As recent machine translation models are mostly based on the attention-based neural machine translation (NMT), many well-known models such as Transformer or bidirectional encoder representations from Transformers (BERT) have been proposed. Along with algorithmic advancements, hardware acceleration methods for those attention-based neural machine translation models have also been introduced. However, the size of the parameters for attention-based NMT is also becoming larger to guarantee the satisfactory machine translation quality. Among various weights, linearization weights ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{Q}$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{K}$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{V}$ </tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$W^{O}$ </tex-math></inline-formula> ) account for a non-negligible portion (by up to 30%) among the entire parameters in the modern NMT models. In this paper, we propose a method for linearization weight compression and near-memory hardware decoder for fast and in-situ weight decompression. Our weight compression method exploits the fixed-point quantization along with Huffman coding which is selectively applied depending on the weight value distribution. Our hardware decoder decompresses the Huffman-coded weights near-memory to minimize the weight decoding latency. Our compression method shows 4.9–10.0 compression ratio with small NMT score drops across the five widely used attention-based NMT models (Transformer, Transformer-XL-base, Transformer-XL-large, BERT-base, and BERT-large). In addition, due to the reduced linearization weight size, our proposed method with near-memory decoding enables multi-head attention (MHA) execution latency reduction by 11.8%, on average, as compared to the baseline when considering the weight loading and initialization. In terms of the memory data transfer energy consumption, our proposed method leads to a memory energy saving of 16.1%, on average, as compared to the baseline.

Issues and Perspectives on The Electronic Translation of Shipping Incoterms: FOB as a case study

The international commercial terms are officially known as Incoterms. These trade rules, established by ICC (the International Chamber of Commerce) are written and pronounced in English. Mastering their use is necessary to circumvent any confusion in the meaning, and to avoid any misunderstanding between the contracting parties. These incoterms are universally agreed upon, except for the United States where a difference in their meaning is noticed, mainly the incoterm FOB (Free on Board). Although they are mandated as multinational business terms, they do not regulate the transfer of ownership of the merchandise nor the sale price. Despite the difference in their meanings, it should be noted that they constitute broad lines for commercial exchanges between countries. Some of their advantages are: The distribution of costs related to export and import, the distribution of documents between the seller and the buyer, and the conditions for the delivery of goods. They are stated to enable trading partners to reach markets throughout the world. To apply incoterms for overseas shipment, maritime transport companies need to make sure that their staff understand them clearly to avoid any costly mistakes in the process of transactions. Collaboration with educational institutions and training courses are required. Both current and future Algerian professionals in the field need to be assisted on how to use incoterms. This research paper deals with the use of shipping incoterms when training the students, who are specialized in maritime trade, for a future profession as a maritime leader or dealer. As the incoterms are basically written and pronounced in English, the use of a translating method, that allows them to identify and understand the meaning of linguistic terms, is required. Bearing in mind the fact that artificial intelligence covers a wide range of outputs nowadays, it has been necessary to ask the following question: Could the T M (translation machine) provide the students a faithful translation of incoterms? In an attempt to answer this question, the following hypotheses were formulated: Teaching EOP (English for occupational purposes) in the Algerian maritime sector focuses on translating the international commercial terms. Machine translation provides a faithful meaning of words and sentences. As the software performs translation for a specific language or domain, it produces a high-quality output. To check these hypotheses, it is essential to go through the MT translation method for it is mostly used by the learners as it is free from any charges. The MT (Machine Translation) method is what a computer program produces. To fulfil a translation, the text is simply entered or pasted in and the languages are selected. Google Translation is a well-known example of this type of translation. The use of this software enables the students to substitute mechanically and quickly words from one language to another. It exempts them from any human linguistic intervention. Though it provides free and instant translations, it produces incoherent wording. The present study reveals that different electronic sites do not give the same translation for the incoterm FOB. One way of evaluating the quality of machine translation is when the human translator reviews the output to ensure that it is linguistically correct. This review is referred to as the post-editing machine translation. When adopting this method, a good quality of translation is achieved; grammatical errors are fixed, unclear and confusing wording and mistranslations are cleared up.

OUTLINE OF A DIDACTIC FRAMEWORK FOR COMBINED DATA LITERACY AND MACHINE TRANSLATION LITERACY TEACHING

This paper outlines a didactic framework for combined data literacy and machine translation (MT) literacy teaching for translation and specialised communication students. The framework is being developed in the context of the DataLitMT project, a publicly funded project at the Institute of Translation and Multilingual Communication at TH Köln – University of Applied Sciences, Germany, which develops didactic resources for teaching data literacy in its translation-specific form of MT literacy to students of BA and MA programmes in translation and specialised communication studies. After discussing the high relevance of machine trans-lation literacy and data literacy in professional translation contexts, the paper introduces the DataLitMT project and discusses a framework of professional MT literacy and an MT-oriented data literacy framework, which form the two theoretical pillars of the project. Also, the interface between MT literacy and data literacy will be illustrated by showing how specific (sub)dimensions of data literacy can be mapped to relevant (sub)dimensions of professional MT literacy. Finally, the paper presents some preliminary didactic resources of the DataLitMT project – concerned with social biases in MT, with MT training data preparation and with automatic MT quality evaluation – and discusses how these resources can be used to teach specific (sub)dimensions of data literacy and professional MT literacy to students in the fields of translation/specialised communication studies. Keywords: data literacy, professional machine translation literacy, neural machine translation, translation didactics, DataLitMT

Improving the Quality of Machine Translation Using the Reverse Model

Improving the Quality of Machine Translation Using the Reverse Model

Reconhecimento Automático Multilingue de Entidades Mencionadas em Diversos Domínios, para Efeitos de Anonimização de Tradução Automática

The following article describes the research developed at Unbabel, a Portuguese Machine-Translation start-up, that combines Machine Translation (MT) with human post-edition with a focus on customer service content. With the work carried out within a real multilingual AI powered, human-refined, MT industry, we aim to contribute to furthering MT quality and good-practices, by exposing the importance of having continuously in development, robust Named Entity Recognition systems for General Data Protection Regulation (GDPR) compliance. We will report three different experiments, resulting from a shared work with Unbabel´s linguists and Unbabel´s Artificial Intelligence (AI) engineering team, matured over a year. The first experiment focused on developing a methodology for the identification and annotation of domain-specific Named Entities (NEs) for the Food-Industry. The devised methodology allows the construction of gold standards for building domain specific NER systems and can be applied for a myriad of different domains. With the implementation of the designed method, we were able to identify the following domain-specific NEs set: Restaurant Names; Restaurant Chains; Dishes; Beverage, Ingredients. The second and third experiments explored the possibilities of constructing, in a semi-automatically way, multilingual NER gold standards for different domains and language pairs, using aligners that project Named Entities across a parallel corpus. Both experiments made it possible to benchmark four different open-source aligners (SimAlign; Fastalign; AwesomeAlign; Eflomal), allowing to identify the one with better performance and, simultaneously, validate the aforementioned approach. This work should be taken as a statement of multidisciplinary, proving and validating the much-needed articulation between different scientific fields that compose and characterize the area of Natural Language Processing (NLP).

On the Translation Strategies of Human-computer Interaction Based on Machine Translation

With the rapid development of language service industry and artificial intelligence technology, machine translation plays a more prominent role in the translation industry. Human-computer interaction translation greatly improves the speed and quality of translation, and pre-translation editing and post-translation editing are two important links and manifestations in human-computer interaction collaborative translation. On the basis of summarizing machine translation problems, this paper proposes translation strategies including replacing well-translated terms in advance, rewriting, addition, omission, and shift via pre-editing and post-editing, which greatly improves the quality of machine translation.

Man or machine? Comparing the difficulty of human translation versus neural machine translation post-editing

ABSTRACT This study aims to compare neural machine translation (NMT) post-editing and human translation in terms of task difficulty while considering source text (ST) complexity and machine translation (MT) quality levels, two factors that have been rarely examined in previous comparison studies. Data were obtained from 60 trainee translators concerning the perceived and objective difficulties of post-editing and human translation tasks and the participants’ performance. It was found that (1) the difficulty of the NMT post-editing task, compared to human translation, was significantly influenced by both NMT quality and ST complexity; the difficulty of the post-editing task was significantly lower than that of the human translation task only in the case of high-quality NMT paired with complex ST, while the results were mixed for other interactions between NMT quality and ST complexity levels; (2) no strong correlations were found between the participants’ perceived difficulty and the measurements of objective difficulty and task performance for both post-editing and human translation tasks. Practical and research implications were discussed.