Neural Machine Translation Systems Research Articles

Instantaneous Inversion of Airborne Electromagnetic Data Based on Deep Learning

AbstractThe airborne electromagnetic (AEM) method is becoming an effective means for subsurface electrical property reconstruction with the merits of terrain adaptability and acquisition efficiency. However, instantaneous inversion of time‐domain AEM data is still a challenge, owing to the huge amount of data. Inspired by Google's neural machine translation system, we develop a fast inversion operator guided by deep learning to translate time‐domain AEM measurements directly into subsurface resistivity structures. Trained by synthetic data, our system shows impressive adaptability to field observations and strong robustness against noise disturbance. Applied to the AEM data set acquired by the U.S. Geological Survey in Leach Lake Basin, CA, USA, our system successfully delivers results in seconds for a common PC from more than 740,000 AEM soundings. The inverted structures clearly delineate the geometries of the lake, surrounding mountains and faults. The inversion operator can support instantaneous subsurface resistivity reconstruction for AEM observations.

Unsupervised English Intelligent Machine Translation in Wireless Network Environment

Researchers suggest unsupervised English machine translation to address the absence of parallel corpus in English translation. Unsupervised pretraining techniques, denoising autoencoders, back translation, and shared latent representation mechanisms are used to simulate the translation task using just monolingual corpora. This paper uses pseudo-parallel data to construct unsupervised neural machine translation (NMT) and dissimilar language pair analysis. This paper firstly analyzes the low performance of unsupervised translation on dissimilar language pairs from three aspects: bilingual word embedding quality, shared words, and word order. And artificial shared word replacement and preordering strategies are proposed to increase the shared words between dissimilar language pairs and reduce the difference in their syntactic structure, thereby improving the translation performance on dissimilar language pairs. The denoising autoencoder and shared latent representation mechanism in unsupervised English machine translation are only required in the early stage of training, and learning the shared latent representation limits the further improvement of performance in different directions. While training the denoising autoencoder by repeatedly altering the training data slows down the convergence of the model, this is especially true for divergent languages. This paper presents an unsupervised NMT model based on pseudo-parallel data to address this issue. It trains two standard supervised neural machine translation models using the pseudo-parallel corpus generated by the unsupervised neural machine translation system, which enhances translation performance and speeds convergence. Finally, the English intelligent translation model is deployed in the wireless network server, and users can access it through the wireless network.

Investigating Contextual Influence in Document-Level Translation

Current state-of-the-art neural machine translation (NMT) architectures usually do not take document-level context into account. However, the document-level context of a source sentence to be translated could encode valuable information to guide the MT model to generate a better translation. In recent times, MT researchers have turned their focus to this line of MT research. As an example, hierarchical attention network (HAN) models use document-level context for translation prediction. In this work, we studied translations produced by the HAN-based MT systems. We examined how contextual information improves translation in document-level NMT. More specifically, we investigated why context-aware models such as HAN perform better than vanilla baseline NMT systems that do not take context into account. We considered Hindi-to-English, Spanish-to-English and Chinese-to-English for our investigation. We experimented with the formation of conditional context (i.e., neighbouring sentences) of the source sentences to be translated in HAN to predict their target translations. Interestingly, we observed that the quality of the target translations of specific source sentences highly relates to the context in which the source sentences appear. Based on their sensitivity to context, we classify our test set sentences into three categories, i.e., context-sensitive, context-insensitive and normal. We believe that this categorization may change the way in which context is utilized in document-level translation.

Improving English-to-Indian Language Neural Machine Translation Systems

Most Indian languages lack sufficient parallel data for Machine Translation (MT) training. In this study, we build English-to-Indian language Neural Machine Translation (NMT) systems using the state-of-the-art transformer architecture. In addition, we investigate the utility of back-translation and its effect on system performance. Our experimental evaluation reveals that the back-translation method helps to improve the BLEU scores for both English-to-Hindi and English-to-Bengali NMT systems. We also observe that back-translation is more useful in improving the quality of weaker baseline MT systems. In addition, we perform a manual evaluation of the translation outputs and observe that the BLEU metric cannot always analyse the MT quality as well as humans. Our analysis shows that MT outputs for the English–Bengali pair are actually better than that evaluated by BLEU metric.

On the Use of Morpho-Syntactic Description Tags in Neural Machine Translation with Small and Large Training Corpora

With the transition to neural architectures, machine translation achieves very good quality for several resource-rich languages. However, the results are still much worse for languages with complex morphology, especially if they are low-resource languages. This paper reports the results of a systematic analysis of adding morphological information into neural machine translation system training. Translation systems presented and compared in this research exploit morphological information from corpora in different formats. Some formats join semantic and grammatical information and others separate these two types of information. Semantic information is modeled using lemmas and grammatical information using Morpho-Syntactic Description (MSD) tags. Experiments were performed on corpora of different sizes for the English–Slovene language pair. The conclusions were drawn for a domain-specific translation system and for a translation system for the general domain. With MSD tags, we improved the performance by up to 1.40 and 1.68 BLEU points in the two translation directions. We found that systems with training corpora in different formats improve the performance differently depending on the translation direction and corpora size.

Optimization of English Machine Translation by Deep Neural Network under Artificial Intelligence.

To improve the function of machine translation to adapt to global language translation, the work takes deep neural network (DNN) as the basic theory, carries out transfer learning and neural network translation modeling, and optimizes the word alignment function in machine translation performance. First, the work implements a deep learning translation network model for English translation. On this basis, the neural machine translation model is designed under transfer learning. The random shielding method is introduced to implement the language training model, and the machine translation is slightly adjusted as the goal of transfer learning, thereby improving the semantic understanding ability in translation performance. Meanwhile, the work design introduces the method of word alignment optimization and optimizes the performance of word alignment in the transformer system by using word corpus. The experimental results show that the proposed method reduces the average alignment error rate by 8.1%, 24.4%, and 22.1% in EnRo (English-Roman), EnGe (English-German), and EnFr (English-French), respectively, compared with the previous algorithms. Compared with the designed optimization method, the word alignment error rate is lower than that of traditional methods. The modeling and optimization method is feasible, which can effectively solve the problems of insufficient information utilization, large parameter scale, and difficult storage in the process of machine translation. Additionally, it provides a feasible idea and direction for the optimization and improvement in neural machine translation (NMT) system.

How are neural machine-translated Chinese-to-English short stories constructed and cohered? An exploratory study based on theme-rheme structure

Using theme-rheme structure in Systemic Functional Linguistics as an analytical framework, this study explores how neural machine-translated Chinese-to-English literary texts are organized and developed in terms of discourse coherence. To this end, we built a comparable translational corpus containing English translations of two modes, viz. neural machine translation (NMT) and human translation, of modern and contemporary Chinese short stories as source texts. The translated texts were annotated with themes, rhemes and their progressions throughout the texts. By analyzing and contrasting their thematic structure and progression patterns, we found that, compared with human translations, 1) NMT texts feature “pseudo-coherence” characterized by ill-connected information fragments linked by overusing and; 2) NMT system fails to adapt the organization of its textual elements to a given purpose and is less competent in maintaining discourse coherence due to a lack of smooth connection to the preceding sentence; 3) NMT system produces static and poorly interconnected texts that read as disjointed as a list consisting of unrelated items by recurrently resorting to a constant progression pattern rather than a dynamic linear progression characteristic of human translation. Hence, we suggest that theme-rheme structure be employed to assess the construction and coherence of machine-translated texts and be incorporated into machine translation models to improve the coherence of their products.

Hierarchical Triple Model of Hybrid Neural Machine Translation

For more than a decade, PMT and SMT models have dominated the field of machine translation, and neural machine translation has emerged as a new paradigm for machine translation by the 2015. Neural machine translation provides a simple modeling mechanism that is easy to use in practice and science. Thus, it does not require concepts such as word ranking, a key component of the statistical machine translation. While this simplicity may be seen as an advantage, on the other hand, the lack of careful spelling is to lose control of the translation. Even tough, the neural machine translation is more flexible in terms of translations that don’t exactly match the training data. This provides more opportunities for such models, but exempts translation from pre-determined restrictions. Failure to connect specific words can make it difficult to connect the target words you create to the original word. The widespread use of neural machine translation system has the advantage of allowing users to translate certain terms and translate uneducated data to a certain extent. In some cases, however, the structure and the grammar boundary of a sentence is often distorted. The paper is intended to address issues such as the control of neural machine translation, more accurate translation of unidentified data, the accuracy of sentence structure and grammar boundaries. To solve this problem, modern translation theory led to the hybrid model of machine translation. Our model is expansion of this hybrid model with a sentence and a grammar boundary. We named this model as hierarchical triple model (HTM).

Creativity in translation

AbstractThis article presents the results of a study involving the translation of a short story by Kurt Vonnegut from English to Catalan and Dutch using three modalities: machine-translation (MT), post-editing (PE) and translation without aid (HT). Our aim is to explore creativity, understood to involve novelty and acceptability, from a quantitative perspective. The results show that HT has the highest creativity score, followed by PE, and lastly, MT, and this is unanimous from all reviewers. A neural MT system trained on literary data does not currently have the necessary capabilities for a creative translation; it renders literal solutions to translation problems. More importantly, using MT to post-edit raw output constrains the creativity of translators, resulting in a poorer translation often not fit for publication, according to experts.

Improving Machine Translation Quality with Denoising Autoencoder and Pre-Ordering

The problems in machine translation are related to the characteristics of a family of languages, especially syntactic divergences between languages. In the translation task, having both source and target languages in the same language family is a luxury that cannot be relied upon. The trained models for the task must overcome such differences either through manual augmentations or automatically inferred capacity built into the model design. In this work, we investigated the impact of multiple methods of differing word orders during translation and further experimented in assimilating the source languages syntax to the target word order using pre-ordering. We focused on the field of extremely low-resource scenarios. We also conducted experiments on practical data augmentation techniques that support the reordering capacity of the models through varying the target objectives, adding the secondary goal of removing noises or reordering broken input sequences. In particular, we propose methods to improve translat on quality with the denoising autoencoder in Neural Machine Translation (NMT) and pre-ordering method in Phrase-based Statistical Machine Translation (PBSMT). The experiments with a number of English-Vietnamese pairs show the improvement in BLEU scores as compared to both the NMT and SMT systems.

Some Translation Studies informed suggestions for further balancing methodologies for machine translation quality evaluation

Abstract This article intends to contribute to the current debate on the quality of neural machine translation (NMT) vs. (professional) human translation quality, where recently claims concerning (super)human performance of NMT systems have emerged. The article will critically analyse some current machine translation (MT) quality evaluation methodologies employed in studies claiming such performance of their MT systems. This analysis aims to identify areas where these methodologies are potentially biased in favour of MT and hence may overvalue MT performance while undervaluing human translation performance. Then, the article provides some Translation Studies informed suggestions for improving or debiasing these methodologies in order to arrive at a more balanced picture of MT vs. (professional) human translation quality.

Lexical Diversity in Statistical and Neural Machine Translation

Neural machine translation systems have revolutionized translation processes in terms of quantity and speed in recent years, and they have even been claimed to achieve human parity. However, the quality of their output has also raised serious doubts and concerns, such as loss in lexical variation, evidence of “machine translationese”, and its effect on post-editing, which results in “post-editese”. In this study, we analyze the outputs of three English to Slovenian machine translation systems in terms of lexical diversity in three different genres. Using both quantitative and qualitative methods, we analyze one statistical and two neural systems, and we compare them to a human reference translation. Our quantitative analyses based on lexical diversity metrics show diverging results; however, translation systems, particularly neural ones, mostly exhibit larger lexical diversity than their human counterparts. Nevertheless, a qualitative method shows that these quantitative results are not always a reliable tool to assess true lexical diversity and that a lot of lexical “creativity”, especially by neural translation systems, is often unreliable, inconsistent, and misguided.

Knowledge Distillation: A Method for Making Neural Machine Translation More Efficient

Neural machine translation (NMT) systems have greatly improved the quality available from machine translation (MT) compared to statistical machine translation (SMT) systems. However, these state-of-the-art NMT models need much more computing power and data than SMT models, a requirement that is unsustainable in the long run and of very limited benefit in low-resource scenarios. To some extent, model compression—more specifically state-of-the-art knowledge distillation techniques—can remedy this. In this work, we investigate knowledge distillation on a simulated low-resource German-to-English translation task. We show that sequence-level knowledge distillation can be used to train small student models on knowledge distilled from large teacher models. Part of this work examines the influence of hyperparameter tuning on model performance when lowering the number of Transformer heads or limiting the vocabulary size. Interestingly, the accuracy of these student models is higher than that of the teachers in some cases even though the student model training times are shorter in some cases. In a novel contribution, we demonstrate for a specific MT service provider that in the post-deployment phase, distilled student models can reduce emissions, as well as cost purely in monetary terms, by almost 50%.

Enriching the transfer learning with pre-trained lexicon embedding for low-resource neural machine translation

Most State-Of-The-Art (SOTA) Neural Machine Translation (NMT) systems today achieve outstanding results based only on large parallel corpora. The large-scale parallel corpora for high-resource languages is easily obtainable. However, the translation quality of NMT for morphologically rich languages is still unsatisfactory, mainly because of the data sparsity problem encountered in Low-Resource Languages (LRLs). In the low-resource NMT paradigm, Transfer Learning (TL) has been developed into one of the most efficient methods. It is difficult to train the model on high-resource languages to include the information in both parent and child models, as well as the initially trained model that only contains the lexicon features and word embeddings of the parent model instead of the child languages feature. In this work, we aim to address this issue by proposing the language-independent Hybrid Transfer Learning (HTL) method for LRLs by sharing lexicon embedding between parent and child languages without leveraging back translation or manually injecting noises. First, we train the High-Resource Languages (HRLs) as the parent model with its vocabularies. Then, we combine the parent and child language pairs using the oversampling method to train the hybrid model initialized by the previously parent model. Finally, we fine-tune the morphologically rich child model using a hybrid model. Besides, we explore some exciting discoveries on the original TL approach. Experimental results show that our model consistently outperforms five SOTA methods in two languages Azerbaijani (Az) and Uzbek (Uz). Meanwhile, our approach is practical and significantly better, achieving improvements of up to 4.94 and 4.84 BLEU points for low-resource child languages Az ! Zh and Uz ! Zh, respectively.

Neural Models for Measuring Confidence on Interactive Machine Translation Systems

Reducing the human effort performed with the use of interactive-predictive neural machine translation (IPNMT) systems is one of the main goals in this sub-field of machine translation (MT). Prior works have focused on changing the human–machine interaction method and simplifying the feedback performed. Applying confidence measures (CM) to an IPNMT system helps decrease the number of words that the user has to check through the translation session, reducing the human effort needed, although this supposes losing a few points in the quality of the translations. The effort reduction comes from decreasing the number of words that the translator has to review—it only has to check the ones with a score lower than the threshold set. In this paper, we studied the performance of four confidence measures based on the most used metrics on MT. We trained four recurrent neural network (RNN) models to approximate the scores from the metrics: Bleu, Meteor, Chr-f, and TER. In the experiments, we simulated the user interaction with the system to obtain and compare the quality of the translations generated with the effort reduction. We also compare the performance of the four models between them to see which of them obtains the best results. The results achieved showed a reduction of 48% with a Bleu score of 70 points—a significant effort reduction to translations almost perfect.

Floating-gate based PN blending optoelectronic synaptic transistor for neural machine translation

Neural machine translation, which has an encoder-decoder framework, is considered to be a feasible way for future machine translation. Nevertheless, with the fusion of multiple languages and the continuous emergence of new words, most current neural machine translation systems based on von Neumann’s architecture have seen a substantial increase in the number of devices for the decoder, resulting in high-energy consumption rate. Here, a multilevel photosensitive blending semiconductor optoelectronic synaptic transistor (MOST) with two different trapping mechanisms is firstly demonstrated, which exhibits 8 stable and well distinguishable states and synaptic behaviors such as excitatory postsynaptic current, short-term memory, and long-term memory are successfully mimicked under illumination in the wavelength range of 480–800 nm. More importantly, an optical decoder model based on MOST is successfully fabricated, which is the first application of neuromorphic device in the field of neural machine translation, significantly simplifying the structure of traditional neural machine translation system. Moreover, as a multi-level synaptic device, MOST can further reduce the number of components and simplify the structure of the codec model under light illumination. This work first applies the neuromorphic device to neural machine translation, and proposes a multi-level synaptic transistor as the based cell of decoding module, which would lay the foundation for breaking the bottleneck of machine translation.

Analysis of Neural Machine Translation KANGRI Language by Unsupervised and Semi Supervised Methods

ABSTRACT It is very challenging to work with low resource languages pairs as monolingual and parallel dataset do not exist or exist in a very small amount. Furthermore, there is a lack of digitization of the available written resources. This work provides a comparison and analysis of the neural machine translation system for low resource definitely endangered, Kangri (ISO 639-3xnr) language using unsupervised and semi supervised methods. For this a shared encoder with back translation machine translation system for both unsupervised and semi-supervised learning techniques and a language model with denoising autoencoder that uses fully unsupervised learning technique has been used. Kangri which is an Indo-Aryan language has Devanagari () script same as Hindi. The translation task is further complicated by the fact that Kangri is a morphologically rich language, and it does not have well defined linguistic rules. To remove out of vocabulary problem we have used different technique and in finally, we have provided the comparison of results by taking the different evaluation metrics which shows that semi supervised translation with semi supervised cross lingual word embedding has highest score as compared to other translation models.

SBSim: A Sentence-BERT Similarity-Based Evaluation Metric for Indian Language Neural Machine Translation Systems

Machine translation (MT) outputs are widely scored using automatic evaluation metrics and human evaluation scores. The automatic evaluation metrics are expected to be easily computable and a reflection of human evaluation. Traditional string-based metrics such as BLEU, ChrF++ scores, are widely used to evaluate MT systems, but fail to account for synonyms that appear in the state-of-the-art neural machine translation (NMT) systems, owing to their inability to evaluate paraphrases. While similarity-based metrics such as Yisi, BERTScore address this issue, these metrics need to be modified to better evaluate morphologically rich Indian languages such as, Tamil and Hindi. The current work proposes a novel and individual sentence-BERT based similarity (SBSim) metric, that makes use of a paraphrase-BERT model and sentence-level embedding to evaluate NMT outputs. The effectiveness of the BLEU, ChrF++, Yisi, BERTScore, and the proposed SBSim are evaluated on English-to-Tamil and English-to-Hindi NMT outputs. The sentence-level metric correlation of the proposed SBSim metric with respect to human scores is observed to outperform the existing metrics with a correlation of 0.9123 and 0.9052 for English-to-Tamil and English-to-Hindi NMT systems, respectively. Further, the average metric correlation of the SBSim metric is also observed to be the highest with a value of 0.9801 and 0.9836 for these NMT systems, respectively. The proposed metric is also evaluated on WMT2020 dataset and reports the highest correlation of 0.7129 with the human scores.

Pre-editing English news texts for machine translation into Russian

The paper discusses the possible advantages of pre-editing English news texts for machine translation into Russian. Pre-editing is defined as a process of adapting source text in order to reach a better quality of machine translation. Two case studies were conducted: one in 2021 and the other one in 2022. During case studies texts from bbc.com were chosen, pre-edited and translated using neural machine translation systems. Analysing common pre-editing techniques and their impact on the result of machine translation in terms of certain error patterns we conclude that in most cases pre-editing helps to eliminate a number of errors, improve the overall quality of machine output and reduce the amount of time and efforts needed for post-editing machine translation. The conducted case study also showed that despite the fact, that machine translation systems are constantly developing and changing, it is possible to identify common peculiarities of machine translation regarding a certain style or type of text and certain language pair, analyse the error patterns and find the appropriate pre-editing techniques that will be applicable for the most of machine translation systems for many months and years. Pre-editing does not guarantee the high quality of the translation itself, but together with post-editing it allows reaching results similar or equal to human translation saving a translator’s time and efforts.

Neural machine translation model combining dependency syntax and LSTM

For the problem of the lack of linguistic knowledge in the neural machine translation model, which is called Transformer, and the insufficient flexibility of positional encoding, this paper introduces the dependency syntax analysis and the long short-term memory network. The source language syntactic structure information is constructed in the neural machine translation system, and the more accurate position information is obtained by using the memory characteristics of LSTM. Experiments show that using the improved model improves by 1.23 BLEU points in the translation task of the IWSLT14 Chinese-English language pair.