Statistical Translation Models Research Articles

Automatic translation from English to Amazigh using transformer learning

Due to the lack of parallel data, to our knowledge, no study has been conducted on the Amazigh-English language pair, despite the numerous machine translation studies completed between major European language pairs. We decided to utilize the neural machine translation (NMT) method on a parallel corpus of 137,322 sentences. The attention-based encoder-decoder architecture is used to construct statistical machine translation (SMT) models based on Moses, as well as NMT models using long short-term memory (LSTM), gated recurrent units (GRU), and transformers. Various outcomes were obtained for each strategy after several simulations: 80.7% accuracy was achieved using the statistical approach, 85.2% with the GRU model, 87.9% with the LSTM model, and 91.37% with the transformer.

Design of intelligent module design for humanoid translation robot by combining the deep learning with blockchain technology

To accelerate the deep application of deep learning in text data processing, an English statistical translation system is established and applied to the question answering of humanoid robot. Firstly, the model of machine translation based on recursive neural network is implemented. A crawler system is established to collect English movie subtitle data. On this basis, an English subtitle translation system is designed. Then, combined with sentence embedding technology, the Particle Swarm Optimization (PSO) algorithm of meta-heuristic algorithm is adopted to locate the defects of translation software. A translation robot automatic question and answer interactive module is constructed. Additionally, the hybrid recommendation mechanism based on personalized learning is built using blockchain technology. Finally, the performance of translation model and software defect location model is evaluated. The results show that the Recurrent Neural Network (RNN) embedding algorithm has certain effect of word clustering. RNN embedded model has a strong ability to process short sentences. The strongest translated sentences are between 11 and 39 words long, while the weakest translated sentences are between 71 and 79 words long. Therefore, the model must strengthen the processing of long sentences, especially character—level input. The average sentence length is much longer than word-level input. The model based on PSO algorithm shows good accuracy in different data sets. This model averages better performance on Tomcat, standard widget toolkits, and Java development tool datasets than other comparison methods. The average reciprocal rank and average accuracy of the weight combination of PSO algorithm are very high. Moreover, this method is greatly affected by the dimension of the word embedding model, and the 300-dimension word embedding model has the best effect. To sum up, this study proposes a good statistical translation model for humanoid robot English translation, which lays the foundation for intelligent interaction between humanoid robots.

Improved Unsupervised Statistical Machine Translation via Unsupervised Word Sense Disambiguation for a Low-Resource and Indic Languages

ABSTRACT Besides word order, word choice is a key stumbling block for machine translation (MT) in morphologically rich languages due to homonyms and polysemous difficulties. On the other hand, un-translated/improperly translated words are a severe issue for Statistical Machine Translation (SMT) models. The quantity of parallel training corpus has limited unsupervised SMT (USMT) systems. Still, current research lines have successfully trained SMT systems in an unsupervised manner using monolingual data alone. However, there is still a need to enhance the translation quality of the MT output due to unaligned and improperly sensed words. This problem is addressed by incorporating unsupervised Word Sense Disambiguation (WSD) into the decoding phase of USMT. The work provided a compendium of SMT systems for five translation tasks, i.e. En→Indic languages for the WMT test dataset and evaluated on BLEU and METEOR evaluation metrics. The studies were performed on En→Hi, En→Kn, En→Ta, En→Te, and En→Be tasks and showed an improvement in BLEU points by 2.3, 2.68, 0.78, 2.32, and 1.79, respectively, and METEOR points by 1.07, 1.34, 0.72, 0.693, and 1.191, respectively, over the baseline model.

Improving Neural Machine Translation by Transferring Knowledge from Syntactic Constituent Alignment Learning

Statistical machine translation (SMT) models rely on word-, phrase-, and syntax-level alignments. But neural machine translation (NMT) models rarely explicitly learn the phrase- and syntax-level alignments. In this article, we propose to improve NMT by explicitly learning the bilingual syntactic constituent alignments. Specifically, we first utilize syntactic parsers to induce syntactic structures of sentences, and then we propose two ways to utilize the syntactic constituents in a perceptual (not adversarial) generator-discriminator training framework. One way is to use them to measure the alignment score of sentence-level training examples, and the other is to directly score the alignments of constituent-level examples generated with an algorithm based on word-level alignments from SMT. In our generator-discriminator framework, the discriminator is pre-trained to learn constituent alignments and distinguish the ground-truth translation from the fake ones, while the generative translation model is fine-tuned to receive the alignment knowledge and to generate translations that best approximate the true ones. Experiments and analysis show that the learned constituent alignments can help improve the translation results.

Translational equivalence in Statistical Machine Translation or meaning as co-occurrence

In this paper, we will describe the current state-of-the-art of Statistical Machine Translation (SMT), and reflect on how SMT handles meaning. Statistical Machine Translation is a corpus-based approach to MT: it de-rives the required knowledge to generate new translations from corpora. General-purpose SMT systems do not use any formal semantic representa-tion. Instead, they directly extract translationally equivalent words or word sequences – expressions with the same meaning – from bilingual parallel corpora. All statistical translation models are based on the idea of word alignment, i.e., the automatic linking of corresponding words in parallel texts. The first generation SMT systems were word-based. From a linguistic point of view, the major problem with word-based systems is that the mean-ing of a word is often ambiguous, and is determined by its context. Current state-of-the-art SMT-systems try to capture the local contextual dependen-cies by using phrases instead of words as units of translation. In order to solve more complex ambiguity problems (where a broader text scope or even domain information is needed), a Word Sense Disambiguation (WSD) module is integrated in the Machine Translation environment.

Generating Chinese Classical Poems with Statistical Machine Translation Models

This paper describes a statistical approach to generation of Chinese classical poetry and proposes a novel method to automatically evaluate poems. The system accepts a set of keywords representing the writing intents from a writer and generates sentences one by one to form a completed poem. A statistical machine translation (SMT) system is applied to generate new sentences, given the sentences generated previously. For each line of sentence a specific model specially trained for that line is used, as opposed to using a single model for all sentences. To enhance the coherence of sentences on every line, a coherence model using mutual information is applied to select candidates with better consistency with previous sentences. In addition, we demonstrate the effectiveness of the BLEU metric for evaluation with a novel method of generating diverse references.

Hybrid System Combination Framework for Uyghur–Chinese Machine Translation

Both the statistical machine translation (SMT) model and neural machine translation (NMT) model are the representative models in Uyghur–Chinese machine translation tasks with their own merits. Thus, it will be a promising direction to combine the advantages of them to further improve the translation performance. In this paper, we present a hybrid framework of developing a system combination for a Uyghur–Chinese machine translation task that works in three layers to achieve better translation results. In the first layer, we construct various machine translation systems including SMT and NMT. In the second layer, the outputs of multiple systems are combined to leverage the advantage of SMT and NMT models by using a multi-source-based system combination approach and the voting-based system combination approaches. Moreover, instead of selecting an individual system’s combined outputs as the final results, we transmit the outputs of the first layer and the second layer into the final layer to make a better prediction. Experiment results on the Uyghur–Chinese translation task show that the proposed framework can significantly outperform the baseline systems in terms of both the accuracy and fluency, which achieves a better performance by 1.75 BLEU points compared with the best individual system and by 0.66 BLEU points compared with the conventional system combination methods, respectively.

Improving Transformer‐Based Neural Machine Translation with Prior Alignments

Transformer is a neural machine translation model which revolutionizes machine translation. Compared with traditional statistical machine translation models and other neural machine translation models, the recently proposed transformer model radically and fundamentally changes machine translation with its self‐attention and cross‐attention mechanisms. These mechanisms effectively model token alignments between source and target sentences. It has been reported that the transformer model provides accurate posterior alignments. In this work, we empirically prove the reverse effect, showing that prior alignments help transformer models produce better translations. Experiment results on Vietnamese‐English news translation task show not only the positive effect of manually annotated alignments on transformer models but also the surprising outperformance of statistically constructed alignments reinforced with the flexibility of token‐type selection over manual alignments in improving transformer models. Statistically constructed word‐to‐lemma alignments are used to train a word‐to‐word transformer model. The novel hybrid transformer model improves the baseline transformer model and transformer model trained with manual alignments by 2.53 and 0.79 BLEU, respectively. In addition to BLEU score, we make limited human judgment on translation results. Strong correlation between human and machine judgment confirms our findings.

Analyzing Subword Techniques to Improve English to Sinhala Neural Machine Translation

Neural machine translation (NMT) is a remarkable approach which performs much better than the Statistical machine translation (SMT) models when there is an abundance of parallel corpus. However, vanilla NMT is primarily based upon word-level with a fixed vocabulary. Therefore, low resource morphologically rich languages such as Sinhala are mostly affected by the out of vocabulary (OOV) and Rare word problems. Recent advancements in subword techniques have opened up opportunities for low resource communities by enabling open vocabulary translation. In this paper, we extend our recently published state-of-the-art EN-SI translation system using the transformer and explore standard subword techniques on top of it to identify which subword approach has a greater effect on English Sinhala language pair. Our models demonstrate that subword segmentation strategies along with the state-of-the-art NMT can perform remarkably when translating English sentences into a rich morphology language regardless of a large parallel corpus.

Research on Statistical Model of Japanese-Chinese Name Translation for Japanese Kana

In view of the lack of bilingual resources in Japanese-Chinese translation, and the naming entity of Chinese character naming entities, especially Japanese pseudonyms, it is difficult to use, so the Japanese-Chinese pure pseudonym translation is a statistical translation model. The model has been applied so far and there have been many problems. The first is limited by the size and quality of the parallel corpus, and the second is the lower accuracy of the translation. Based on this, the paper proposes a statistical model of Japanese pseudonymous Japanese name translation based on inductive learning method. This method can extract Japanese and Chinese named entities from Japanese and Chinese corpora, and convert them into Roman alphabet and Pinyin sequence. Example screening after similarity calculation. Then, the instance-based induction learning method is used to automatically obtain the Japanese-Chinese transliteration rule base of the named entity, and iteratively reconstructs the transliteration rule base through feedback learning. Through experimental research, it is found that the Japanese-Han name translation statistical model method for Japanese kana is simple and efficient, and the translation accuracy is overcome while overcoming the dependence on bilingual data.

Adaptation in Statistical Machine Translation for Low-resource Domains in English-Vietnamese Language

Adaptation in Statistical Machine Translation for Low-resource Domains in English-Vietnamese Language

Revisiting Back-Translation for Low-Resource Machine Translation Between Chinese and Vietnamese

Back-translation (BT) has been widely used and become one of standard techniques for data augmentation in Neural Machine Translation (NMT), BT has proven to be beneficial for improving the performance of translation effectively, especially for low-resource scenarios. While most previous works related to BT mainly focus on European languages with high relatedness, few of them study less-related languages in other areas around the world. In this paper, we choose the language pair with less relatedness in Asia: Chinese and Vietnamese, to investigate the impacts of BT on extremely low-resource machine translation between them. We first discuss the similarities and differences between the two languages, then evaluate and compare the effects of different sizes of back-translated data on NMT and Statistical Machine Translation (SMT) models for Chinese-Vietnamese and Vietnamese-Chinese, with both character-based and word-based settings, and conduct further analysis on the translation outputs from several aspects. Some conclusions from previous works are partially confirmed and we also draw some new findings and conclusions, which are beneficial to understand BT further and deeper for translation between less-related low-resource languages.

Research on statistical machine translation model based on deep neural network

With the increase of translation demand, the advancement of information technology, the development of linguistic theories and the progress of natural language understanding models in artificial intelligence research, machine translation has gradually gained worldwide attention. However, at present, machine translation research still has problems such as insufficient bilingual data and lack of effective feature representation, which affects the further improvement of key modules of machine translation such as word alignment, sequence adjustment and translation modelling. The effect of machine translation is still unsatisfactory. As a new machine learning method, deep neural network can automatically learn abstract feature representation and establish a complex mapping relationship between input and output signals, which provides a new idea for statistical machine translation research. Firstly, the multi-layer neural network and the undirected probability graph model are combined, and the similarity and context information of vocabulary are effectively utilized to model the word alignment more fully, and the word alignment model named NNWAM is constructed. Secondly, the low dimension will be used. The feature representation is combined with other features into a linearly ordered pre-ordering model to construct the pre-ordering model named NNPR. Finally, the word alignment model and the pre-ordering model are combined in the same deep neural network framework to form DNNAPM, a statistical machine translation model based on deep neural networks. The experimental results show that the statistical machine translation model based on deep neural network has better effect, faster convergence and better reliability than the comparison model algorithm.

Incorporating Source-Side Phrase Structures into Neural Machine Translation

Neural machine translation (NMT) has shown great success as a new alternative to the traditional Statistical Machine Translation model in multiple languages. Early NMT models are based on sequence-to-sequence learning that encodes a sequence of source words into a vector space and generates another sequence of target words from the vector. In those NMT models, sentences are simply treated as sequences of words without any internal structure. In this article, we focus on the role of the syntactic structure of source sentences and propose a novel end-to-end syntactic NMT model, which we call a tree-to-sequence NMT model, extending a sequence-to-sequence model with the source-side phrase structure. Our proposed model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. We have empirically compared the proposed model with sequence-to-sequence models in various settings on Chinese-to-Japanese and English-to-Japanese translation tasks. Our experimental results suggest that the use of syntactic structure can be beneficial when the training data set is small, but is not as effective as using a bi-directional encoder. As the size of training data set increases, the benefits of using a syntactic tree tends to diminish.

Research on Mongolian-Chinese machine translation based on the end-to-end neural network

With the development of natural language processing and neural machine translation, the neural machine translation method of end-to-end (E2E) neural network model has gradually become the focus of research because of its high translation accuracy and strong semantics of translation. However, there are still problems such as limited vocabulary and low translation loyalty, etc. In this paper, the discriminant method and the Conditional Random Field (CRF) model were used to segment and label the stem and affixes of Mongolian in the preprocessing stage of Mongolian-Chinese bilingual corpus. Aiming at the low translation loyalty problem, a decoding model combining Convolution Neural Network (CNN) and Gated Recurrent Unit (GRU) was constructed. The target language decoding was performed by using the GRU. A global attention model was used to obtain the bilingual word alignment information in the process of bilingual word alignment processing. Finally, the quality of the translation was evaluated by Bilingual Evaluation Understudy (BLEU) values and Perplexity (PPL) values. The improved model yields a BLEU value of 25.13 and a PPL value of [Formula: see text]. The experimental results show that the E2E Mongolian-Chinese neural machine translation model was improved in terms of translation quality and semantic confusion compared with traditional statistical methods and machine translation models based on Recurrent Neural Networks (RNN).

Automatic Generation of Children's Songs Based on Machine Statistic Learning

In this paper, the automatic generation of children's songs is studied. First of all, according to the number of key lyrics submitted by the songwriter, statistical machine learning is used to expand and obtain more theme-related lyrics, and then the first sentence is generated through the language model automatically. On this basis, the subsequent sentences are generated through the statistical machine learning translation method. In the process of the generation, the statistical machine learning is used to expand the conception of the song, so as to get richer sentence candidates. The main features and contributions of the study are: Firstly, the statistical machine learning translation is put forward as the theoretical basis, the preceding and next sentence relationship of children's songs are mapped into the relation of the source language and target language in the statistical translation model, and the machine statistics learning translation model is designed with the integration of the domain knowledge of songs. Secondly, the statistical machine learning is used in the generation process to expand the lyrics words, thereby enhancing the theme and conception of the song. The experimental results have confirmed the effectiveness of the proposed method.

Machine Translation and Parallel Corpora: The Case of English-to-Chinese Translation of Relative Clauses

This paper compares translations done by professional human translators (as manifested in the Babel English-Chinese Parallel Corpus) and machine translations produced by Google Translate (with English as the source language and Simplified Chinese as the target language). Capitalizing on the branching direction of these two languages, we investigated translation strategies of restrictive relative clauses by focusing on which-relatives. The specific objectives of the study were to (a) extract relevant data from a parallel corpus; (b) assess machine translation quality; and (c) highlight the similarities and dissimilarities between corpus outputs and machine translation outputs. Of the 147 test materials, 115 Google Translate outputs (78.2%) were rated as ‘successful’ or ‘acceptable’. No significant differences were found between the degree of success in machine translations and linguistic factors (e.g., the active or passive voice in relative clauses). This finding confirms that linguistic knowledge is not required when using statistical machine translation (SMT) such as Google Translate. Another noteworthy finding was that Google Translate did not use the least frequently occurring translation strategy in the parallel corpus. This is not surprising given that SMT systems greatly rely on parallel corpora for training statistical translation models.

Integrating Shallow Syntactic Labels in the Phrase-Boundary Translation Model

Using a novel rule labeling method, this article proposes a hierarchical model for statistical machine translation. The proposed model labels translation rules by matching the boundaries of target side phrases with the shallow syntactic labels including POS tags and chunk labels on the target side of the training corpus. The boundary labels are concatenated if there is no label for the whole target span. Labeling with the classes of boundary words on the target side phrases has been previously proposed as a phrase-boundary model which can be considered as the base form of our model. In the extended model, the labeler uses a POS tag if there is no chunk label in one boundary. Using chunks as phrase labels, the proposed model generalizes the rules to decrease the model sparseness. The sparseness is a more important issue in the language pairs with a lot of differences in the word order because they have less number of aligned phrase pairs for extraction of rules. The extended phrase-boundary model is also applicable for low-resource languages having no syntactic parser. Some experiments are performed with the proposed model, the base phrase-boundary model, and variants of Syntax Augmented Machine Translation (SAMT) in translation from Persian and German to English as source and target languages with different word orders. According to the results, the proposed model improves the translation performance in the quality and decoding time aspects. Using BLEU as our metric, the proposed model has achieved a statistically significant improvement of about 0.5 point over the base phrase-boundary model.

TermFinder: log-likelihood comparison and phrase-based statistical machine translation models for bilingual terminology extraction

Bilingual termbanks are important for many natural language processing applications, especially in translation workflows in industrial settings. In this paper, we apply a log-likelihood comparison method to extract monolingual terminology from the source and target sides of a parallel corpus. The initial candidate terminology list is prepared by taking all arbitrary n-gram word sequences from the corpus. Then, a well-known statistical measure (the Dice coefficient) is employed in order to remove any multi-word terms with weak associations from the candidate term list. Thereafter, the log-likelihood comparison method is applied to rank the phrasal candidate term list. Then, using a phrase-based statistical machine translation model, we create a bilingual terminology with the extracted monolingual term lists. We integrate an external knowledge source—the Wikipedia cross-language link databases—into the terminology extraction (TE) model to assist two processes: (a) the ranking of the extracted terminology list, and (b) the selection of appropriate target terms for a source term. First, we report the performance of our monolingual TE model compared to a number of the state-of-the-art TE models on English-to-Turkish and English-to-Hindi data sets. Then, we evaluate our novel bilingual TE model on an English-to-Turkish data set, and report the automatic evaluation results. We also manually evaluate our novel TE model on English-to-Spanish and English-to-Hindi data sets, and observe excellent performance for all domains.

A Neural Approach to Source Dependence Based Context Model for Statistical Machine Translation

In statistical machine translation, translation prediction considers not only the aligned source word itself but also its source contextual information. Learning context representation is a promising method for improving translation results, particularly through neural networks. Most of the existing methods process context words sequentially and neglect source long-distance dependencies. In this paper, we propose a novel neural approach to source dependence-based context representation for translation prediction. The proposed model is capable of not only encoding source long-distance dependencies but also capturing functional similarities to better predict translations (i.e., word form translations and ambiguous word translations). To verify our method, the proposed mode is incorporated into phrase-based and hierarchical phrase-based translation models, respectively. Experiments on large-scale Chinese-to-English and English-to-German translation tasks show that the proposed approach achieves significant improvement over the baseline systems and outperforms several existing context-enhanced methods.