Machine Translation Models Research Articles

Corpus Processing of Multi-Word Discourse Markers for Advanced Learners

Aim/Purpose. The most crucial aspects of teaching a foreign language to more advanced learners are building an awareness of discourse modes, how to regulate discourse, and the pragmatic properties of discourse components. However, in different languages, the connections and structure of discourse are ensured by different linguistic means which makes matters complicated for the learner. Background. By uncovering regularities in a foreign language and comparing them with patterns in one’s own tongue, the corpus research method offers the student unique opportunities to acquire linguistic knowledge about discourse markers. This paper reports on an investigation of the functions of multi-word discourse markers. Methodology. In our research, we combine the alignment model of the phrase-based statistical machine translation and manual treatment of the data in order to examine English multi-word discourse markers and their equivalents in Lithuanian and Hebrew translations by researching their changes in translation. After establishing the full list of multi-word discourse markers in our generated parallel corpus, we research how the multi-word discourse markers are treated in translation. Contribution. Creating a parallel research corpus to identify multi-word expressions used as discourse markers, analyzing how they are translated into Lithuanian and Hebrew, and attempting to determine why the translators made the choices add value to corpus-driven research and how to manage discourse. Findings. Our research proves that there is a possible context-based influence guiding the translation to choose a particle or other lexical item integration in Lithuanian or Hebrew translated discourse markers to express the rhetorical domain which could be related to the so-called phenomenon of “over-specification.” Recommendations for Practitioners. The comparative examination of discourse markers provides language instructors and translators with more specific information about the roles of discourse markers. Recommendations for Researchers. Understanding the multifunctionality of discourse markers provides new avenues for discourse marker application in translation research. Impact on Society. The current study may be a useful method to strengthen students’ language awareness and analytic skills and is particularly important for students specializing in English philology or translation. Beyond the empirical research, an extensive parallel data resource has been created to be openly used. Future Research. It should be noted that the observed phenomenon of “over-specification” could be analyzed further in future research.

Pipeline Signed Japanese Translation Using PBSMT and Transformer in a Low-Resource Setting

We propose a novel pipeline method for translating signed Japanese sentences into written Japanese. Sign languages often suppress functional words such as particles, and most words are not morphologically inflected as they are in spoken languages. Our method explicitly compares and contrasts the two languages and divides the translation process into two tasks: first, it translates glosses into lemmatized Japanese words or phrases, followed by complementing particles and conjugating predicates such as verbs, auxiliary verbs, and adjectives. Our method is especially effective when the size of the parallel corpus is very limited and costly to obtain, but there are plenty of monolingual corpora for the target. Specifically, our method first uses phrase-based statistical machine translation (PBSMT) to map sign glosses to corresponding Japanese words or phrases, and then employs a transformer-based neural machine translation (NMT) model trained with a monolingual corpus to refine the output in the first translation. Experimental results show that our pipeline method outperforms direct PBSMT and competitive NMT models with data augmentation, including back-translation and transfer learning in a low-resource setting with a corpus size on the order of 104 words.

Revolutionizing Translation with AI: Unravelling Neural Machine Translation and Generative Pre-Trained Large Language Models

Revolutionizing Translation with AI: Unravelling Neural Machine Translation and Generative Pre-Trained Large Language Models

A Deep Learning-Based Intelligent Quality Detection Model for Machine Translation

A Deep Learning-Based Intelligent Quality Detection Model for Machine Translation

Neural Machine Translation by Fusing Key Information of Text

When the Transformer proposed by Google in 2017, it was first used for machine translation tasks and achieved the state of the art at that time. Although the current neural machine translation model can generate high quality translation results, there are still mistranslations and omissions in the translation of key information of long sentences. On the other hand, the most important part in traditional translation tasks is the translation of key information. In the translation results, as long as the key information is translated accurately and completely, even if other parts of the results are translated incorrect, the final translation results’ quality can still be guaranteed. In order to solve the problem of mistranslation and missed translation effectively, and improve the accuracy and completeness of long sentence translation in machine translation, this paper proposes a key information fused neural machine translation model based on Transformer. The model proposed in this paper extracts the keywords of the source language text separately as the input of the encoder. After the same encoding as the source language text, it is fused with the output of the source language text encoded by the encoder, then the key information is processed and input into the decoder. With incorporating keyword information from the source language sentence, the model’s performance in the task of translating long sentences is very reliable. In order to verify the effectiveness of the method of fusion of key information proposed in this paper, a series of experiments were carried out on the verification set. The experimental results show that the Bilingual Evaluation Understudy (BLEU) score of the model proposed in this paper on the Workshop on Machine Translation (WMT) 2017 test dataset is higher than the BLEU score of Transformer proposed by Google on the WMT2017 test dataset. The experimental results show the advantages of the model proposed in this paper.

Retracted: Research on the Construction of a Bidirectional Neural Network Machine Translation Model Fused with Attention Mechanism

Retracted: Research on the Construction of a Bidirectional Neural Network Machine Translation Model Fused with Attention Mechanism

An Assistant System for Translation Flipped Classroom

To achieve the goal of training translators that meet the current social needs, the innovation of translation teaching methods is necessary. Studies have proven that students in flipped classrooms (FCs) have greater performance than students in traditional classrooms. However, the preparation time for FCs could be three times higher than that of traditional classrooms, which leads to the reluctance of teachers to conduct FCs. Machine translation (MT) is believed to be a useful tool to improve the translation efficiency of human translators. However, in practice, teachers found that many students cannot work with MT effectively. To solve the above problems, this paper designs a Translation Flipped Classroom Assistance System (TFCAS) based on cloud computing and MT. A parameter is proposed to measure students’ ability to translate evaluation. TFCAS has reduced the burden of teachers in the FC mode and helped students become accustomed to working with MT. Application data stored in the MySQL database, such as sentence pairs, will be used to optimize the neural machine translation model we developed for the system. The system makes MT and the training of translators support each other’s sustainable development and conforms to the trend of deepening teaching reform.

A Rule Matrix Machine Translation (RMMT) Model for the Phrases Having Homographs That Cause Difficulties in Machine Translation

Doğal diller tam olarak matematiksel formüllerle ifade edilemediği ve sosyal kabullere dayanan sözcük veya ifadeler barındırdıkları için makine çevirilerinde bazı anlam belirsizliği durumlarıyla karşılaşılabilmektedir. Bu belirsizlik durumuna en uygun örnek eş yazımlı sözcük(ler) bulunduran ifadelerdir. Çalışmada bu ifadelerin kural tabanlı bir çeviri modeli ile doğru şekilde İngilizceden Türkçeye çevrilebilmesinin mümkün olup olmadığı sorusu ele alınmıştır. Bu maksatla, Kural Matrisli Makine Çevirisi (KMMÇ) olarak adlandırılan bir model geliştirilmiştir. Önerilen modelin başarı ölçümleri için Google, Bing Yandex ve KMMÇ çıktıları çevirmen değerlendirmesi yoluyla karşılaştırılmış; değerlendirmelerde KMMÇ'nin, eş yazımlı bulunduran ifadelerde dikkate değer başarı elde ettiği görülmüştür.

Deep Tasks Summarization for Comprehending Mixed Tasks in a Commit

In Version Control System (VCS), a developer frequently uploads multiple tasks such as adding features, code refactoring, and fixing bugs, into a single commit and crumbles each task’s summary when writing a commit message. It causes code readers to feel challenged in understanding the developer’s past tasks within the commit history. To resolve this issue, we propose an automatic approach to generating a task summary to help comprehend multiple mixed tasks in a commit and developed tool support named Task summary Generator (TsGen). In our approach, we use the commit with a single task as input and identify the task to sort its elements sequentially. Then we generate feature vectors from each sorted element to train the Neural Machine Translation (NMT) model. Based on the trained NMT model, we generate the feature vector from each task of a commit with multiple tasks and put each of them into the model to provide the task summary. In evaluation, we compared the performance of TsGen with two existing methods for nine open-source projects. As a result, TsGen outperformed CoDiSum and Jiang’s NMT by 52.08% and 28.07% in BiLingual Evaluation Understudy (BLEU) scores. In addition, the human evaluation was carried out to demonstrate that TsGen helps understand mixed tasks in a commit and gained a 0.27 higher preference than the actual commit message.

AmericasNLI: Machine translation and natural language inference systems for Indigenous languages of the Americas.

Little attention has been paid to the development of human language technology for truly low-resource languages-i.e., languages with limited amounts of digitally available text data, such as Indigenous languages. However, it has been shown that pretrained multilingual models are able to perform crosslingual transfer in a zero-shot setting even for low-resource languages which are unseen during pretraining. Yet, prior work evaluating performance on unseen languages has largely been limited to shallow token-level tasks. It remains unclear if zero-shot learning of deeper semantic tasks is possible for unseen languages. To explore this question, we present AmericasNLI, a natural language inference dataset covering 10 Indigenous languages of the Americas. We conduct experiments with pretrained models, exploring zero-shot learning in combination with model adaptation. Furthermore, as AmericasNLI is a multiway parallel dataset, we use it to benchmark the performance of different machine translation models for those languages. Finally, using a standard transformer model, we explore translation-based approaches for natural language inference. We find that the zero-shot performance of pretrained models without adaptation is poor for all languages in AmericasNLI, but model adaptation via continued pretraining results in improvements. All machine translation models are rather weak, but, surprisingly, translation-based approaches to natural language inference outperform all other models on that task.

Mongolian-Chinese Machine Translation Based on Graph Neural Network

Most of the current Mongolian-Chinese neural machine translation relies on a sequence-to-sequence-based encoder-decoder, which does not effectively utilize the syntactic information of sentences as well as the hierarchical information of the language. To solve this problem, a graph-to-sequence-based encoder-decoder is constructed by introducing graph neural networks, making better use of syntactic information and semantic knowledge of sentences. We will build a Mongolian dependency tree library and design a densely connected graph convolutional neural network (D-GCN) based on GCN combined with a densely connected network (DenseNet). Experiments are conducted to compare with the sequence-to-sequence based Mongolian-Chinese neural machine translation model and the variant model of graph neural network, and the results verify the advantages of the D-GCN-based Mongolian-Chinese neural machine translation model in translation performance.

An End-to-End Contrastive Self-Supervised Learning Framework for Language Understanding

Abstract Self-supervised learning (SSL) methods such as Word2vec, BERT, and GPT have shown great effectiveness in language understanding. Contrastive learning, as a recent SSL approach, has attracted increasing attention in NLP. Contrastive learning learns data representations by predicting whether two augmented data instances are generated from the same original data example. Previous contrastive learning methods perform data augmentation and contrastive learning separately. As a result, the augmented data may not be optimal for contrastive learning. To address this problem, we propose a four-level optimization framework that performs data augmentation and contrastive learning end-to-end, to enable the augmented data to be tailored to the contrastive learning task. This framework consists of four learning stages, including training machine translation models for sentence augmentation, pretraining a text encoder using contrastive learning, finetuning a text classification model, and updating weights of translation data by minimizing the validation loss of the classification model, which are performed in a unified way. Experiments on datasets in the GLUE benchmark (Wang et al., 2018a) and on datasets used in Gururangan et al. (2020) demonstrate the effectiveness of our method.

Application of Machine Learning Combined with Wireless Network in Design of Online Translation System

The importance of translation services has become increasingly prominent with the acceleration of economic globalization. Compared with human translation, machine translation is cheaper and faster, and therefore more suitable for the current era. The current mainstream machine translation method is neural machine translation, which employs machine methods to train on parallel corpora and create translation models. Research into neural machine translation has yielded a wealth of information. Learning and generalization abilities of neural networks have substantially enhanced the effectiveness of neural machine translation. This work applies machine learning and wireless network technology to build an online translation system for real-time translation. First, this work proposes a multigranularity feature fusion method based on a directed acyclic graph, which uses a directed acyclic graph to fuse different granularities as input and obtain a position representation. Secondly, this paper improves the Transformer model and proposes multigranularity position encoding and multigranularity self-attention. Then, on the basis of multigranularity features as input, this work introduces dynamic word vectors to improve the word embedding module, and uses the ELMo model to obtain dynamic word vector embeddings. Finally, this work builds a multigranularity feature-dynamic word vector machine translation model with above strategy, deploys it on server. Users can upload the content to be translated and download the translated content through the wireless network and realize an online translation system based on machine learning and wireless network.

Analysis on the Recent Trends in Machine Translation

Machine translation is to translate one language into another language, which has undergone a great evolution. The model of machine translation has been continuously improved, aiming to make the translation effect closer to the artificial translation. This article briefly summarizes the development history of machine translation, and introduces the main models of each stage of development. The initial machine translation mode is the Rule Based Machine Translation (RBMT) and Statistical Machine Translation (SMT). Recent mainstream translation approach enables Neural Machine Translation (NMT). It includes the input and the output, attention mechanism, and BLEU evaluation method. On this basis, there are also many expansion and innovation models, such as GPKD and other models to improve the evaluation effect. In general, machine translation can replace a part of human translation. However, it cannot completely replace human beings, because of the different human thinking and machine logic. People and machines have to cooperate with each other to improve the common efficiency.

Rectifying Ill-Formed Interlingual Space: A Framework for Zero-Shot Translation on Modularized Multilingual NMT

The multilingual neural machine translation (NMT) model can handle translation between more than one language pair. From the perspective of industrial applications, the modularized multilingual NMT model (M2 model) that only shares modules between the same languages is a practical alternative to the model that shares one encoder and one decoder (1-1 model). Previous works have proven that the M2 model can benefit from multiway training without suffering from capacity bottlenecks and exhibits better performance than the 1-1 model. However, the M2 model trained on English-centric data is incapable of zero-shot translation due to the ill-formed interlingual space. In this study, we propose a framework to help the M2 model form an interlingual space for zero-shot translation. Using this framework, we devise an approach that combines multiway training with a denoising autoencoder task and incorporates a Transformer attention bridge module based on the attention mechanism. We experimentally show that the proposed method can form an improved interlingual space in two zero-shot experiments. Our findings further extend the use of the M2 model for multilingual translation in industrial applications.

Building User-oriented Personalized Machine Translator based on User-Generated Textual Content

Machine Translation (MT) has been a very useful tool to assist multilingual communication and collaboration. In recent years, by taking advantage of the exciting developments of neural networks and deep learning, the accuracy and speed of machine translation have been continuously improved. However, most machine translation methods and systems are data-driven. They tend to select a consensus response represented in training data, while a user's preferred linguistic style, which is important for translation comprehension and user experience, is ignored. For this problem, we aim to build a user-oriented personalized machine translation model in this paper. The model aims to learn each user's linguistic style from the textual content that is generated by her/him (User-Generated Textual Content, UGTC) in social media context and generate personalized translation results utilizing several state-of-the-art deep learning techniques like Transformer and pre-training. We also implemented a user-oriented personalized machine translator using Weibo as a case of the source of UGTC to provide a systematical implementation scheme of a user-oriented personalized machine translation system based on our model. The translator was evaluated by automatic evaluation in combination with human evaluation. The results suggest that our model can generate more personalized, natural and lively translation results and enhance the comprehensibility of translation results, which makes its generations more preferred by users versus general translation results.

TOWARDS CROSS-ATTENTION PRE-TRAINING IN NEURAL MACHINE TRANSLATION

The advent of pre-train techniques and large language models has significantly leveraged the performance of many natural language processing (NLP) tasks. However, pre-trained language models for neural machine translation remain a challenge as little information about the interaction of the language pair is learned. In this paper, we explore several studies trying to define a training scheme to pre-train the cross-attention module between the encoder and the decoder by using the large-scale monolingual corpora independently. The experiments show promising results, proving the effectiveness of using pre-trained language models in neural machine translation.

A Comparative Study of AI translation - Google Translate and Papago

Since the mid 2010s, there have been a number of studies on the reliability of the translation capability of AI translation programs. Shin (2017) notes how machine translation systems have developed from a translation researcher’s perspective. Johnson et al. (2016) proposes a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Kim (2017) discusses how human spiritual aspect can be kept intact in the AI technology-dominated society. Kwak et al. (2020) conducted a study on AI translation focusing on proverb translation. Besides, Bahdanau et al. (2015) and Chung (2020) conducted research on machine translation. Following Park (2017, 2018)’s seminal research on the accuracy of the translation capability of Google Translate, this study investigates whether Google Translate and Papago, a translation program developed in Korea, have acquired native-like ability in translating sentences with semantically ambiguous Korean expressions such as –ya hayssta (had to vs was supposed to), -su issessta (could vs was able to), construction involving negation and sayngkakhata (think), mitta (believe), nukkita (feel) + embedded clause, and –cul alassta (knew or thought).

Cross-Context Accurate English Translation Method Based on the Machine Learning Model

The era of big data and cloud computing has come, communication between different languages is becoming more and more common, and the barriers between languages are becoming more and more prominent. As the most important means to overcome language barriers, machine translation will play an increasingly important role in modern society. The previous machine translation technology has more or less disadvantages. The accuracy of translation is too low, which is a huge bottleneck hindering the further development of machine translation technology. Therefore, based on this, we can consider modeling the cross-context accurate English translation model based on the machine translation model and rely on the working principle of machine learning. This experiment shows that the translation accuracy of our method reaches 94.2%, which is higher than 39.5% of the benchmark method. This shows that the method in this paper can reduce the influence of other factors, ensure the accuracy of cross-context English translation to a certain extent, and meet the performance improvement requirements of the English translation system.

An Exploratory Study on Utilising the Web of Linked Data for Product Data Mining

The Linked Open Data practice has led to a significant growth of structured data on the Web. While this has created an unprecedented opportunity for research in the field of Natural Language Processing, there is a lack of systematic studies on how such data can be used to support downstream NLP tasks. This work focuses on the e-commerce domain and explores how we can use such structured data to create language resources for product data mining tasks. To do so, we process billions of structured data points in the form of RDF n-quads, to create multi-million words of product-related corpora that are later used in three different ways for creating language resources: training word-embedding models, continued pre-training of BERT-like language models, and training machine translation models that are used as a proxy to generate product-related keywords. These language resources are then evaluated in three downstream tasks, product classification, linking, and fake review detection using an extensive set of benchmarks. Our results show word embeddings to be the most reliable and consistent method to improve the accuracy on all tasks (with up to 6.9% points in macro-average F1 on some datasets). Contrary to some earlier studies that suggest a rather simple but effective approach such as building domain-specific language models by pre-training using in-domain corpora, our work serves a lesson that adapting these methods to new domains may not be as easy as it seems. We further analyse our datasets and reflect on how our findings can inform future research and practice.