Neural Machine Translation Research Articles

NLP-reliant Neural Machine Translation techniques used in smart city applications

For smart city applications, Neural Machine Translation (NMT) methods based on Natural Language Processing (NLP) are crucial as they facilitate information sharing and communication among diverse populations. NLP techniques are used in many domains related to smart cities, such as development and research, business, industries, media, healthcare, and residences and communities. The majority of people in India communicate using their regional languages. The majority of applications used by users in smart cities will mostly accept English as input. These people will be able to interact with these smart city devices in their native tongues more effectively with the help of effective machine translation. Just 10% of Indians use English as their primary language of communication; there are 22 official regional languages in India. So, there is requirement of better machine translation using Natural language processing (NLP). Natural language processing for Indian regional languages has a very long way to go until it surpassing the abilities of existing rich NLP applications and techniques for English language. Machine Translation is technique of Natural Language Processing (NLP) which provides better inter-lingual communication. For low resourced Indian languages effective machine translation systems became important for establishing proper communication. Machine Transliteration is a technique to convert source language into target language using machine. The developed system takes English language as input and then applies machine translation techniques to translate the source language into multiple languages using trained RNN model and multi-lingual search model which search the input word across all the datasets and generate the output into other Indian languages such as Hindi, Tamil. Our approach achieves top performance for English-Hindi language pair and comparable results for other cases.

Machine translation literacy in the legal translation context: a SWOT analysis perspective

ABSTRACT In the past couple of decades, machine translation (MT), whether phrase-based statistical or neural MT, has made considerable progress and has become increasingly common in specialised translation workflows. More studies on MT in legal contexts have been conducted in the past several years, and MT has been and continues to be used in international and supranational institutions that rely on legal translation and whose voluminous collections of translated text have been used to train contemporary systems. The time is ripe for a synthesis of developments in the area that may reinforce our understanding of what we might expect from MT from the perspective of legal translators. This systematic review of legal translation and MT studies teases out potential strengths/weaknesses of MT and opportunities/threats of using it from a legal translator perspective, focusing on lexical and morphosyntactic challenges that may set legal translation apart from other specialised translation areas. A SWOT analysis from this perspective may offer value as a pedagogical framework for fostering domain-specific MT literacy in the legal translation classroom. Students will develop an understanding of what might reasonably be expected from MT in legal translation contexts, while reinforcing their understanding of legal translation challenges.

Novel algorithm machine translation for language translation tool

AbstractFuzzy matching techniques are the presently used methods in translating the words. Neural machine translation and statistical machine translation are the methods used in MT. In machine translator tool, the strategy employed for translation needs to handle large amount of datasets and therefore the performance in retrieving correct matching output can be affected. In order to improve the matching score of MT, the advanced techniques can be presented by modifying the existing fuzzy based translator and neural machine translator. The conventional process of modifying architectures and encoding schemes are tedious process. Similarly, the preprocessing of datasets also involves more time consumption and memory utilization. In this article, a new spider web based searching enhanced translation is presented to be employed with the neural machine translator. The proposed scheme enables deep searching of available dataset to detect the accurate matching result. In addition, the quality of translation is improved by presenting an optimal selection scheme for using the sentence matches in source augmentation. The matches retrieved using various matching scores are applied to an optimization algorithm. The source augmentation using optimal retrieved matches increases the translation quality. Further, the selection of optimal match combination helps to reduce time requirement, since it is not necessary to test all retrieved matches in finding target sentence. The performance of translation is validated by measuring the quality of translation using BLEU and METEOR scores. These two scores can be achieved for the TA‐EN language pairs in different configurations of about 92% and 86%, correspondingly. The results are evaluated and compared with other available NMT methods to validate the work.

Development of system for generating questions, answers, distractors using transformers

The goal of this article is to develop a multiple-choice questions generation system that has a number of advantages, including quick scoring, consistent grading, and a short exam period. To overcome this difficulty, we suggest treating the problem of question creation as a sequence-to-sequence learning problem, where a sentence from a text passage can directly mapped to a question. Our approach is data-driven, which eliminates the need for manual rule implementation. This strategy is more effective and gets rid of potential errors that could result from incorrect human input. Our work on question generation, particularly the usage of the transformer model, has been impacted by recent developments in a number of domains, including neural machine translation, generalization, and picture captioning.

Comparison of Adam Optimization and RMS prop in Minangkabau-Indonesian Bidirectional Translation with Neural Machine Translation

Language is a tool humans use to establish communication. Still, the language used is one language and between regions or nations with their languages. Indonesia is a country that has a diversity of second languages and is the fourth most populous country in the world. It is recorded that Indonesia has nearly 800 regional languages, but research activities in natural language processing are still lacking. Minangkabau is an endangered language spoken by the Minangkabau people in Indonesia's West Sumatra province. According to UNESCO, the Minangkabau language is listed as a language that is "definitely endangered," with only around 5 million speakers worldwide. This study uses neural machine translation (NMT) to create a formula based on this information. Neural machine translation, in contrast to conventional statistical machine translation, intends to build a single neural network that can be built up to achieve the best performance. Because it can simultaneously hold memory for a long time, comprehend complicated relationships in data, and provide information that is very important in determining the outcome of translation, LSTM is one of the most powerful machine-learning techniques for translating languages. The BLUE score is utilized in the NMT evaluation. The test results use 520 Minangkabau sentences, conducting tests based on the number of epochs ranging from 100-1000, resulting in optimization using Adam being better than optimization RMSprop. This is evidenced by the results of the best BLUE-1 score of 0.997816 using 1000 epochs.

한일 인공신경망 기계번역의 정확도 및 오류에 관한 품사별 고찰 - 명사를 중심으로 -

한일 인공신경망 기계번역의 정확도 및 오류에 관한 품사별 고찰 - 명사를 중심으로 -

Leveraging Generative Pre-trained Language Models for Advanced Unsupervised Neural Machine Translation

Leveraging Generative Pre-trained Language Models for Advanced Unsupervised Neural Machine Translation

Implementation of Neural Machine Translation in Translating from Indonesian to Sasak Language

Language translation is part of Natural Language Processing, also known as Machine Translation, which helps the process of learning foreign and regional languages using translation technology in sentence form. In Lombok, there are still people who are not very fluent in Indonesian because Indonesian is generally only used at formal events. This research aimed to develop a translation model from Indonesian to Sasak. The method used was the Neural Machine Translation with the Recurrent Neural Network - Long Short Term Memory architecture and the Word2Vec Embedding with a sentence translation system. The dataset used was a parallel corpus from the Tatoeba Project and other open sources, divided into 80% training and 20% validation data. The result of this research was the application of Neural Machine Translation with the Recurrent Neural Network - Long Short Term Memory algorithm, which could produce a model with an accuracy of 99.6% in training data and 71.9% in test data. The highest ROUGE evaluation metric result obtained on the model was 88%. This research contributed to providing a translation model from Indonesian to Sasak for the local community to facilitate communication and preserve regional language culture.

Towards Understanding Neural Machine Translation with Attention Heads’ Importance

Although neural machine translation has made great progress, and the Transformer has advanced the state-of-the-art in various language pairs, the decision-making process of the attention mechanism, a crucial component of the Transformer, remains unclear. In this paper, we propose to understand the model’s decisions by the attention heads’ importance. We explore the knowledge acquired by the attention heads, elucidating the decision-making process through the lens of linguistic understanding. Specifically, we quantify the importance of each attention head by assessing its contribution to neural machine translation performance, employing a Masking Attention Heads approach. We evaluate the method and investigate the distribution of attention heads’ importance, as well as its correlation with part-of-speech contribution. To understand the diverse decisions made by attention heads, we concentrate on analyzing multi-granularity linguistic knowledge. Our findings indicate that specialized heads play a crucial role in learning linguistics. By retaining important attention heads and removing the unimportant ones, we can optimize the attention mechanism. This optimization leads to a reduction in the number of model parameters and an increase in the model’s speed. Moreover, by leveraging the connection between attention heads and multi-granular linguistic knowledge, we can enhance the model’s interpretability. Consequently, our research provides valuable insights for the design of improved NMT models.

Neural Machine Translation with CARU-Embedding Layer and CARU-Gated Attention Layer

The attention mechanism performs well for the Neural Machine Translation (NMT) task, but heavily depends on the context vectors generated by the attention network to predict target words. This reliance raises the issue of long-term dependencies. Indeed, it is very common to combine predicates with postpositions in sentences, and the same predicate may have different meanings when combined with different postpositions. This usually poses an additional challenge to the NMT study. In this work, we observe that the embedding vectors of different target tokens can be classified by part-of-speech, thus we analyze the Natural Language Processing (NLP) related Content-Adaptive Recurrent Unit (CARU) unit and apply it to our attention model (CAAtt) and embedding layer (CAEmbed). By encoding the source sentence with the current decoded feature through the CARU, CAAtt is capable of achieving translation content-adaptive representations, which attention weights are contributed and enhanced by our proposed L1expNx normalization. Furthermore, CAEmbed aims to alleviate long-term dependencies in the target language through partial recurrent design, performing the feature extraction in a local perspective. Experiments on the WMT14, WMT17, and Multi30k translation tasks show that the proposed model achieves improvements in BLEU scores and enhancement of convergence over the attention-based plain NMT model. We also investigate the attention weights generated by the proposed approaches, which indicate that refinement over the different combinations of adposition can lead to different interpretations. Specifically, this work provides local attention to some specific phrases translated in our experiment. The results demonstrate that our approach is effective in improving performance and achieving a more reasonable attention distribution compared to the state-of-the-art models.

Improving neural machine translation by word translations

Transformer-based neural machine translation (NMT) models have achieved state-of-the-art performance in the machine translation paradigm. These models learn the translation knowledge from the bilingual corpus through the attention mechanism automatically. This differs from the way human translators approach sentence translation, where prior knowledge plays a significant role. Inspired by this, a word translation augmentation (WTA) method is proposed to improve the Transformer-based NMT model. The main steps are as follows: Firstly, constructing the word alignment rules based on the training set. Next, generating the translation rules for source words according to the word alignment rules. Lastly, incorporating the potential translation candidates for each source word into the NMT model during the training and testing procedure. In addition, the WTA method introduces the idea of Mixup for translation candidates of a source word and employs two augmentation strategies to augment the encoder. The results of experiments on several translation tasks with high-resource and low-resource indicate the effectiveness of the proposed method compared with the corresponding strong baseline, and the improvement in BLEU score achieved ranges from 0.42 to 0.63.

Revitalizing Bahnaric Language through Neural Machine Translation: Challenges, Strategies, and Promising Outcomes

The Bahnar, a minority ethnic group in Vietnam with ancient roots, hold a language of deep cultural and historical significance. The government is prioritizing the preservation and dissemination of Bahnar language through online availability and cross-generational communication. Recent AI advances, including Neural Machine Translation (NMT), have transformed translation with improved accuracy and fluency, fostering language revitalization through learning, communication, and documentation. In particular, NMT enhances accessibility for Bahnar language speakers, making information and content more available. However, translating Vietnamese to Bahnar language faces practical hurdles due to resource limitations, particularly in the case of Bahnar language as an extremely low-resource language. These challenges encompass data scarcity, vocabulary constraints, and a lack of fine-tuning data. To address these, we propose transfer learning from selected pre-trained models to optimize translation quality and computational efficiency, capitalizing on linguistic similarities between Vietnamese and Bahnar language. Concurrently, we apply tailored augmentation strategies to adapt machine translation for the Vietnamese-Bahnar language context. Our approach is validated through superior results on bilingual Vietnamese-Bahnar language datasets when compared to baseline models. By tackling translation challenges, we help revitalize Bahnar language, ensuring information flows freely and the language thrives.

Text to video generation via knowledge distillation

<p>Text-to-video generation (T2V) has recently attracted more attention due to the wide application scenarios of video media. However, compared with the substantial advances in text-to-image generation (T2I), the research on T2V remains in its early stage. The difficulty mainly lies in maintaining the text-visual semantic consistency and the video temporal coherence. In this paper, we propose a novel distillation and translation GAN (DTGAN) to address these problems. First, we leverage knowledge distillation to guarantee semantic consistency. We distill text-visual mappings from a well-performing T2I teacher model and transfer it to our DTGAN. This knowledge serves as shared abstract features and high-level constraints for each frame in the generated videos. Second, we propose a novel visual recurrent unit (VRU) to achieve video temporal coherence. The VRU can generate frame sequences as well as process the temporal information across frames. It enables our generator to act as a multi-modal variant of the language model in neural machine translation task, which iteratively predicts the next frame based on the input text and the previously generated frames. We conduct experiments on two synthetic datasets (SBMG and TBMG) and one real-world dataset (MSVD). Qualitative and quantitative comparisons with state-of-the-art methods demonstrate that our DTGAN can generate results with better text-visual semantic consistency and temporal coherence.<strong></strong></p>

Consensus-Based Machine Translation for Code-Mixed Texts

Multilingualism in India is widespread due to its long history of foreign acquaintances. This leads to the presence of an audience familiar with conversing using more than one language. Additionally, due to the social media boom, the usage of multiple languages to communicate has become extensive. Hence, the need for a translation system that can serve the novice and monolingual user is the need of the hour. Such translation systems can be developed by methods such as statistical machine translation and neural machine translation, where each approach has its advantages as well as disadvantages. In addition, the parallel corpus needed to build a translation system, with code-mixed data, is not readily available. In the present work, we present two translation frameworks that can leverage the individual advantages of these pre-existing approaches by building an ensemble model that takes a consensus of the final outputs of the preceding approaches and generates the target output. The developed models were used for translating English-Bengali code-mixed data (written in Roman script) into their equivalent monolingual Bengali instances. A code-mixed to monolingual parallel corpus was also developed to train the preceding systems. Empirical results show improved BLEU and TER scores of 17.23 and 53.18 and 19.12 and 51.29, respectively, for the developed frameworks.

Addressing data scarcity issue for English–Mizo neural machine translation using data augmentation and language model

Low-resource language in machine translation systems poses multiple complications regarding accuracy in translation due to insufficient incorporation of linguistic information. The difference in the linguistic information between the language pair also significantly impacts the dataset creation for improving translation accuracy. Although neural machine translation achieves a state-of-the-art approach, dealing with low-resource language is challenging since it struggled with limited resources. This paper attempts to address the data scarcity problem using augmentation of synthetic parallel sentences, source-target phrase pairs, and language models at the target side for English-to-Mizo and Mizo-to-English translation via transformer-based neural machine translation. We have attained state-of-the-art results for both directions of translation.

Handling syntactic difference in Chinese-Vietnamese neural machine translation

As the typical distant language pair, Chinese and Vietnamese vary widely in syntactic structure, which significantly influences the performance of Chinese-Vietnamese machine translation. To address this problem, we present a simple approach with a pre-reordering model for closing syntactic gaps of the Chinese-Vietnamese language pair. Specifically, we first propose an algorithm for recognizing the modifier inverse, one of the most representative syntactic different in Chinese-Vietnamese language pair. Then we pre-train a pre-reordering model based on the former recognition algorithm and incorporate it into the attention-based translation framework for syntactic different reordering. We conduct empirical studies on Chinese-Vietnamese neural machine translation task, the results show that our approach achieves average improvement of 2.75 BLEU points in translation quality over the baseline model. In addition, the translation fluency can be significantly improved by over 2.44 RIBES points.

Word translation for Indo-Aryan languages using different retrieval techniques

<p>The study of Natural Language Processing has been revolutionized by word embedding, enabling advanced language models to understand and generate human-like text. In this research article, we delve deep into the world of word embedding, aiming to provide a comprehensive exploration of its underlying principles, methodologies, and applications. One important factor that affects many multilingual language processing activities is the word translation or incorporation of bilingual dictionaries. We used bilingual dictionaries or parallel data for translation from one language to another. For this research work, this problem is addressed, and also generating the best cross-lingual word embedding for the different language pairs. So, we are using an aligned document sentence-aligned corpus, or any bilingual dictionary for this research analysis. For the most frequent word, we are assuming that there is an intra-lingual similarity distribution, and both the source and the target corpora have a comparable distribution graph. Additionally, these embeddings are isometric. These cross-lingual word embeddings are used for cross-lingual transfer learning and unsupervised neural machine translation. This research aims to improve the accuracy and efficiency of word translation between different language pairs by employing different retrieval techniques. The study analyzes the effectiveness of these techniques on different language pairs, including English-Hindi, English-Punjabi, English-Gujarati, English-Bengali, and English-Marathi. The research is expected to contribute significantly to the field of language translation by introducing innovative methods and other applications.</p>

Exploring Topics in Domestic Research on Machine Translation through Text Mining

Machine translation (MT) has become increasingly ubiquitous in our daily lives, academia, and education. Learners use MT to understand foreign language texts, and teachers use MT in a variety of courses, such as general English and courses for international students. This study aims to investigate the trends of MT research in Korea and to identify the research topics that have been studied so far. To this end, this study collected 875 domestic MT research papers from the 1970s to the present and extracted the topics of the papers using topic modeling. This study then analyzed the temporal trends of each research topic to examine the changes in research topics and to identify their networks. This study's findings show that the number of research papers and research topics have changed significantly since the release of Google's neural machine translation in late 2016.

A neural machine translation method based on split graph convolutional self-attention encoding

With the continuous advancement of deep learning technologies, neural machine translation (NMT) has emerged as a powerful tool for enhancing communication efficiency among the members of cross-language collaborative teams. Among the various available approaches, leveraging syntactic dependency relations to achieve enhanced translation performance has become a pivotal research direction. However, current studies often lack in-depth considerations of non-Euclidean spaces when exploring interword correlations and fail to effectively address the model complexity arising from dependency relation encoding. To address these issues, we propose a novel approach based on split graph convolutional self-attention encoding (SGSE), aiming to more comprehensively utilize syntactic dependency relationships while reducing model complexity. Specifically, we initially extract syntactic dependency relations from the source language and construct a syntax dependency graph in a non-Euclidean space. Subsequently, we devise split self-attention networks and syntactic semantic self-attention networks, integrating them into a unified model. Through experiments conducted on multiple standard datasets as well as datasets encompassing scenarios related to team collaboration and enterprise management, the proposed method significantly enhances the translation performance of the utilized model while effectively mitigating model complexity. This approach has the potential to effectively enhance communication among cross-language team members, thereby ameliorating collaborative efficiency.

Evaluation of Instagram's Neural Machine Translation for Literary Texts: An MQM-Based Analysis

Evaluation of Instagram's Neural Machine Translation for Literary Texts: An MQM-Based Analysis