Machine Translation Quality Research Articles

A Study on Chinese-English Machine Translation Based on Transfer Learning and Neural Networks

The existing Chinese-English machine translation has problems such as inaccurate word translation and difficult translation of long sentences. To this end, this paper proposes a new machine translation model based on bidirectional Chinese-English translation incorporating translation knowledge and transfer learning, and the components of this model include a recurrent neural network-based translation quality assessment model and a self-focused network-based model. The experimental results demonstrate that our method works better on the dataset of machine translation quality assessment task for Chinese-English translation with more information, and the Pearson correlation coefficient of its quality assessment feature vector (such as word prediction vector representation) is higher.

English-Chinese Machine Translation Model Based on Bidirectional Neural Network with Attention Mechanism

In recent years, with the development of deep learning, machine translation using neural network has gradually become the mainstream method in industry and academia. The existing Chinese-English machine translation models generally adopt the deep neural network architecture based on attention mechanism. However, it is still a challenging problem to model short and long sequences simultaneously. Therefore, a bidirectional LSTM model integrating attention mechanism is proposed. Firstly, by using the word vector as the input data of the translation model, the linguistic symbols used in the translation process are mathematized. Secondly, two attention mechanisms are designed: local attention mechanism and global attention mechanism. The local attention mechanism is mainly used to learn which words or phrases in the input sequence are more important for modeling, while the global attention mechanism is used to learn which layer of expression vector in the input sequence is more critical. Bidirectional LSTM can better fuse the feature information in the input sequence, while bidirectional LSTM with attention mechanism can simultaneously model short and long sequences. The experimental results show that compared with many existing translation models, the bidirectional LSTM model with attention mechanism can effectively improve the quality of machine translation.

Simplification of English and Bengali Sentences for Improving Quality of Machine Translation

Simplification of English and Bengali Sentences for Improving Quality of Machine Translation

Pre-editing English news texts for machine translation into Russian

The paper discusses the possible advantages of pre-editing English news texts for machine translation into Russian. Pre-editing is defined as a process of adapting source text in order to reach a better quality of machine translation. Two case studies were conducted: one in 2021 and the other one in 2022. During case studies texts from bbc.com were chosen, pre-edited and translated using neural machine translation systems. Analysing common pre-editing techniques and their impact on the result of machine translation in terms of certain error patterns we conclude that in most cases pre-editing helps to eliminate a number of errors, improve the overall quality of machine output and reduce the amount of time and efforts needed for post-editing machine translation. The conducted case study also showed that despite the fact, that machine translation systems are constantly developing and changing, it is possible to identify common peculiarities of machine translation regarding a certain style or type of text and certain language pair, analyse the error patterns and find the appropriate pre-editing techniques that will be applicable for the most of machine translation systems for many months and years. Pre-editing does not guarantee the high quality of the translation itself, but together with post-editing it allows reaching results similar or equal to human translation saving a translator’s time and efforts.

Word-Level Quality Estimation for Korean-English Neural Machine Translation

Quality estimation (QE) task aims to predict the machine translation (MT) quality well by referring to the source sentence and its MT output. The various applicability of QE proves the importance of QE research, but the enormous human labor to construct the QE dataset remains a challenge. This study proposes three automatic word-level pseudo-QE data construction strategies using a monolingual or parallel corpus and an external machine translator without human labor. We utilize these individual pseudo-QE datasets to finetune multilingual pretrained language models such as cross-lingual language models (XLM), XLM-RoBERTa, and multilingual BART and comparatively analyze the results. Considering the synthetic dataset creation setup, we attempt to validate the objectivity of the QE model by leveraging four test sets translated by external translators from Google, Amazon, Microsoft, and Systran. As a result, XLM-R-large shows the best performance among mPLMs. We also verify the reliability of the QE model through the close performance gaps between different test sets. To the best of our knowledge, this is the first study to experiment with word-level Korean-English QE.

Автоматизований переклад та його «постмашинне» редагування

The article examines the peculiarities of text editing, in cases the translation is performed by using one of the automated translation systems. The relevance of the study is justified by the widespread use of automated translation programs, on the one hand, and the insufficient quality of machine translations, on the other. The introduction of automated systems into the practice of translation allows the analysis of machine translation texts to be included in the range of research tasks. In the applied aspect, there is a need to develop training courses that highlight the issue of machine translation. The purpose of the article is to define the nature of the tasks facing the editor of machine translation, and to develop general approaches to text editing. The task of the study is to analyze the journalistic text, to identify the nature of errors in the machine translation of phrases and sentences of the analyzed text, and to select methods for editing specific translation units. The objectives of the study also include the identification of typical errors of machine translation, followed by the definition of the post-editing algorithm, as well as the formulation of recommendations for machine translation editing. The study examined examples of machine translation, or rather examples of losses in translation from British English. It has been established that the most typical mistake in the translation of set phrases or set expressions is a word for word translation. The greatest difficulty for a machine translator is the translation of metaphorical expressions and phrases containing realities. At the same time, the degree of difficulty in translating a metaphor can be different. It was noted that post-editing is a mandatory element of working with the texts of machine translation. In automated translation, the machine acts as a supporter of word for word translation, and a human editor as a supporter of free translation. Post-editing of machine translation includes the ability to see logical errors and inconsistencies, the ability to select matches, to offer a descriptive translation, and, if necessary, to add a comment. During the analysis of automated translation, a certain dependence of the error rate on the characteristics of language units, namely, on the nature of lexical bindings, was revealed. In the course of the study, such correlations were established as typical characteristics of the text and the nature of automated translation errors; the nature of machine translation units and the degree of difficulty in editing. In general, it was concluded that the degree of participation of the translator or the linguist in editing the machine translation depends on the typical characteristics of the text.

A Method of Chinese-Vietnamese Bilingual Corpus Construction for Machine Translation

A bilingual corpus is vital for natural language processing problems, especially in machine translation. The larger and better quality the corpus is, the higher the efficiency of the resulting machine translation is. There are two popular approaches to building a bilingual corpus. The first is building one automatically based on resources that are available on the internet, typically bilingual websites. The second approach is to construct one manually. Automated construction methods are being used more frequently because they are less expensive and there are a growing number of bilingual websites to exploit. In this paper, we use automated collection methods for a bilingual website to create a bilingual Chinese-Vietnamese corpus. In particular, the bilingual website we use to collect the data is the website of a multilingual dictionary (https://glosbe.com). We collected the Chinese-Vietnamese corpus from this website that includes more than 400k sentence pairs. We chose 100,000 sentence pairs in this corpus for machine translation experiments. From the corpus, we built five datasets consisting of 20k, 40k, 60k, 80k, and 100k sentence pairs, respectively. In addition, we built five additional datasets, applying word segmentation on the sentences of the original datasets. The experimental results showed that: (1) the quality of the corpus is relatively good with the highest BLEU score of 19.8, although there are still some issues that need to be addressed in future works; (2) the larger the corpus is, the higher the machine translation quality is; and (3) the untokenized datasets help train better translation models than the tokenized datasets.

Machine Translation Quality in Mobile Apps for Text-based Image Translation

With the advancement of mobile applications, now it is possible to perform instant text translation using a smartphone’s camera. Because text translation within images is still a relatively new field of research, it is not surprising that the translation quality of these mobile applications is under-researched. This study aims to determine the image-to-text translation quality in the English to Lithuanian language direction using popular machine translation apps. To classify errors and evaluate the quality of translation, the present study adopts and customizes the Multidimensional Quality Metrics (MQM) framework (Lommel 2014). The obtained results indicate that image-to-text machine translation apps produce exceptionally low-quality translations for the English-Lithuanian language pair. Therefore, the quality of machine translation for low-resource languages such as Lithuanian remains an issue.

Human-Computer Interaction in Translation Activity: Fluency of Machine Translation

Digitalization is one of the key distinctive features of modern environment and social life. Nowadays more and more functions are transferred to the artificial mind. How effective is the replacement of human activity with computer activity? In the given article, this problem is solved by an example of integration of digital technologies into translation activities. It this paper, emphasis is placed on the quality of machine translation (MT) output of legal texts in the language pair English - Slovak. It studies a Criminal Code formulated in the Slovak language which was translated by a human translator into English and consequently via machine translation system Google Translate (GT) back into Slovak. The back-translation - translation of a translated text back into its original language - as a quality assessment tool to detect discrepancies, mistranslations and inevitable differences between the source text and the target text was used. The quality of MT output was evaluated according to Multidimensional Quality Metrics (MQM) standards with the focus on the dimension of Fluency. The multiple comparisons were applied to determine which issues (errors) in Fluency dimension differ from the others. A statistically significant difference is noticed between Agreement and other issues, as well as between Ambiguity and other issues. The errors in Agreement are related to the differences between the languages: English is considered mostly an analytic language, Slovak represents a synthetic language. The issues in the Ambiguity dimension correlate with the type of the text being examined, since legal texts are characterized by relatively complicated wording and numerous terms; moreover, accuracy and unambiguity need to be preserved. Generally, the MT output is able to provide users with basic information about the text. On the other hand, most of the segments need revision and/or correction; in such cases, human intervention and post-editing is necessary.

포스트에디팅 결과물의 정확성 오류 고찰 — AI 학습용 금융/증시 분야 한-영 번역 말뭉치를 대상으로 —

In sharp contrast to great attention to the quality of Machine Translation (MT) raw output, the quality of post-edited output has drawn relatively little attention in Korean translation studies, although some errors in MT output can remain even after post-editing. Against this backdrop, this study sets out to investigate accuracy errors in post-edited output, based on Korean-English parallel translation corpus for AI training released in June 2021 by the National Information Society Agency. For this purpose, 200 parallel sentences with accuracy errors were collected and classified by error type. According to the analysis results, mistranslation errors account for about two-thirds, with the rest in omissions, indicating that quite a number of omissions are still left in post-edited output. While lexical errors ranging from words to clauses are found most frequently in mistranslations, syntax errors represent a surprisingly large portion, with many errors in modifiers and subjects. This study draws attention to quality in MT post-editing, suggesting the need for further investigation into factors affecting the quality of post-edited output.

The challenges of machine translation of academic publications

Clear translation remains a major challenge to better communication and understanding of the international academic literature, despite advances in Machine Translation (MT). Automatic translation systems which captured the detail and the sense of any manuscript in any language for a reader from any other linguistic background would find global applications.In this article, we discuss the current opportunities and constraints to the wider use of machine translation and computer-assisted human translation (CAT). At the present stage of technology development, these instruments offer a number of advantages to specialists working with scientific texts. These include the facility to skim and scan large amounts of information in foreign languages, and to act as digital dictionaries, thesauri and encyclopedias. Word-to-word and phrase-to-phrase translation between many languages and scripts is now well advanced.The availability of modern machine translation has therefore changed the work of specialist scientific translators, placing greater emphasis on more advanced text and sense editing skills. However, machine translation is still challenged by the nuances of language and culture from one society to another, particularly in the freestyle literature of the arts and humanities. Scientific papers are generally much more structured, but the quality of machine translation still largely depends on the quality of the source text. This varies considerably between different scientific disciplines and from one author to another.The most advanced translation systems are making steady progress. It is timely to revisit traditional training programmes in the field of written translation to focus on the development of higher-level research competencies, such as terminology search, and so to make best use of evolving machine translation technologies.More widely, we consider that there is a challenge across the higher education systems in all countries to develop a simple, clear and consistent “international” writing style to assist fast, reliable and low-cost machine translation and hence to advance mutual understanding across the global scientific literature.

Product and Process Analysis of Machine Translation into the Inflectional Language

This study focuses on the influence of quality of Machine Translation (MT) output on a translator’s performance. We analyze the translator’s effort by product analysis and process analysis. The product analysis consists of MT quality evaluation according to the Dynamic Quality Framework; using error typology and the criteria such as fluency and adequacy. We examine translator’s effort from the point of view of typing time, in the context of MT quality—focusing on error rate in language, accuracy, terminology, and style, and also in fluency and adequacy to the source text. We have found that the translator’s performance is influenced by MT quality. The typing time is very closely related to errors in language, accuracy, terminology, and style as well as to fluency and adequacy. We used the Mann-Whitney test to compare the productivity of post-editing of MT with human translation. The results of the study have shown that post-editing—compared to human translation of journalistic text from English into the inflectional Slovak language is more effective.

The Role of Machine Translation Quality Estimation in the Post-Editing Workflow

As Machine Translation (MT) becomes increasingly ubiquitous, so does its use in professional translation workflows. However, its proliferation in the translation industry has brought about new challenges in the field of Post-Editing (PE). We are now faced with a need to find effective tools to assess the quality of MT systems to avoid underpayments and mistrust by professional translators. In this scenario, one promising field of study is MT Quality Estimation (MTQE), as this aims to determine the quality of an automatic translation and, indirectly, its degree of post-editing difficulty. However, its impact on the translation workflows and the translators’ cognitive load is still to be fully explored. We report on the results of an impact study engaging professional translators in PE tasks using MTQE. To assess the translators’ cognitive load we measure their productivity both in terms of time and effort (keystrokes) in three different scenarios: translating from scratch, post-editing without using MTQE, and post-editing using MTQE. Our results show that good MTQE information can improve post-editing efficiency and decrease the cognitive load on translators. This is especially true for cases with low MT quality.

THE TRANSLATION QUALITY PROBLEMS OF MACHINE TRANSLATION SYSTEMS FOR THE KAZAKH LANGUAGE

Kazakh language is related to languages with complex morphology and syntax. Today, most machine translation systems consider Kazakh language too, like Google, Yandex, Prompt, etc. This article describes the errors, shortcomings, and problems of machine translation (MT) into the Kazakh language. To analyze errors in machine translation into the Kazakh language, the most popular electronic translation programs were selected. When translating from Russian and English into Kazakh (and vice versa), various errors may occur, since the Kazakh language is dierent from other languages, and has special characteristics. To compare the results, an empirical method was used, namely, monitoring and testing the translation results of machine translation systems. Considering the results of statistical methods, the rule-based and neural networks based methods in machine translations were also analyzed. The practical signicance of the study lies in the development of recommendations for the identication and elimination of errors when editing the results of MT. The scientic signicance of the study lies in the fact that for the rst time errors and inaccuracies arising from machine translation of the Kazakh language have been systematized. The assessment of the quality of MT is also presented. The research carried out in this article will be used for the post-editing problem in machine translation.

Machine Translation System Based on Deep Learning

The translation fusion of multi-machine translation is a strategy that has been proposed very early to improve the quality of machine translation. It has been fully discussed under the framework of statistical machine translation, and further breakthroughs in performance have entered a bottleneck period. Since the fusion methods of existing methods are mostly limited to surface features and lack effective deep fusion strategies, this paper encodes source language sentences and multiple system translations separately to achieve fusion at the coding level. In the specific decoding, we limit the decoding space, so that the translation model can obtain a higher performance improvement on less data. When increasing the values of the two hyper-parameters α and β, the translation effect can be gradually improved. The performance of the model reaches the maximum when a and b are respectively set to 0.4, and the translation performance will decrease if it continues to increase.

An Oblivious Approach to Machine Translation Quality Estimation

Machine translation (MT) is being used by millions of people daily, and therefore evaluating the quality of such systems is an important task. While human expert evaluation of MT output remains the most accurate method, it is not scalable by any means. Automatic procedures that perform the task of Machine Translation Quality Estimation (MT-QE) are typically trained on a large corpus of source–target sentence pairs, which are labeled with human judgment scores. Furthermore, the test set is typically drawn from the same distribution as the train. However, recently, interest in low-resource and unsupervised MT-QE has gained momentum. In this paper, we define and study a further restriction of the unsupervised MT-QE setting that we call oblivious MT-QE. Besides having no access no human judgment scores, the algorithm has no access to the test text’s distribution. We propose an oblivious MT-QE system based on a new notion of sentence cohesiveness that we introduce. We tested our system on standard competition datasets for various language pairs. In all cases, the performance of our system was comparable to the performance of the non-oblivious baseline system provided by the competition organizers. Our results suggest that reasonable MT-QE can be carried out even in the restrictive oblivious setting.

Эффективность систем машинного перевода в урбанистическом дискурсе

This article aims to analyze the use of Yandex.Translate, an online machine translation system, in translating urban discourse texts on the web. The authors use integrative linguistic-and-pragmatic approach to assess machine translation quality in a global digital setting. The aim is to show the efficiency of a state-of-the-art machine translation system and to investigate its usefulness in practical application. The authors perform a detailed analysis of the Paris city website content, which is automatically translated from French into Russian with Yandex.Translate. The data selection is justified by the absence of official foreign versions of this website, which points to the need of machine translation engines integrated in a web browser. Less than 20% of the analysed machine-translated texts demonstrate high language quality, whereas 60% can be referred to as acceptable – the text preserves the meaning of the source but contains some errors and inaccuracies in the target language. About 20% of the machine-translated text contains blunders, which violate Russian language norms. It causes source text contents distortion and communication failures. In the end, a classification of the system errors is presented. It is also concluded that machine translation would substitute middle-skilled human translators in the future. However, the use of such systems will enforce standardisation and simplification of the target language.

Comparative Analysis of Current Approaches to Quality Estimation for Neural Machine Translation

Quality estimation (QE) has recently gained increasing interest as it can predict the quality of machine translation results without a reference translation. QE is an annual shared task at the Conference on Machine Translation (WMT), and most recent studies have applied the multilingual pretrained language model (mPLM) to address this task. Recent studies have focused on the performance improvement of this task using data augmentation with finetuning based on a large-scale mPLM. In this study, we eliminate the effects of data augmentation and conduct a pure performance comparison between various mPLMs. Separate from the recent performance-driven QE research involved in competitions addressing a shared task, we utilize the comparison for sub-tasks from WMT20 and identify an optimal mPLM. Moreover, we demonstrate QE using the multilingual BART model, which has not yet been utilized, and conduct comparative experiments and analyses with cross-lingual language models (XLMs), multilingual BERT, and XLM-RoBERTa.

Methodology for the Evaluation of Machine Translation Quality

Along with the development and widespread dissemination of translation by artificial intelligence, it is becoming increasingly important to continuously evaluate and improve its quality and to use it as a tool for the modern translator. In our research, we compared five sentences translated from Armenian into Russian and English by Google Translator, Yandex Translator and two models of the translation system of the Armenian company Avromic to find out how effective these translation systems are when working in Armenian. It was necessary to find out how effective it would be to use them as a translation tool and in the learning process by further editing the translation.
 As there is currently no comprehensive and successful method of human metrics for machine translation, we have developed our own evaluation method and criteria by studying the world's most well-known methods of evaluation for automatic translation. We have used the post-editorial distance evaluation criterion as well. In the example of one sentence in the article, we have presented in detail the evaluation process according to the selected and developed criteria. At the end we have presented the results of the research and made appropriate conclusions.

DirectQE: Direct Pretraining for Machine Translation Quality Estimation

Machine Translation Quality Estimation (QE) is a task of predicting the quality of machine translations without relying on any reference. Recently, the predictor-estimator framework trains the predictor as a feature extractor, which leverages the extra parallel corpora without QE labels, achieving promising QE performance. However, we argue that there are gaps between the predictor and the estimator in both data quality and training objectives, which preclude QE models from benefiting from a large number of parallel corpora more directly. We propose a novel framework called DirectQE that provides a direct pretraining for QE tasks. In DirectQE, a generator is trained to produce pseudo data that is closer to the real QE data, and a detector is pretrained on these data with novel objectives that are akin to the QE task. Experiments on widely used benchmarks show that DirectQE outperforms existing methods, without using any pretraining models such as BERT. We also give extensive analyses showing how fixing the two gaps contributes to our improvements.