Example-based Machine Translation Research Articles

COMPARATIVE STUDY OF MACHINE TRANSLATION TECHNIQUES

Machine translation (MT) is an approach to give knowledge and train computer to translate the sentence from one language to another language with the same fluent and meaning. MT is evaluated by the techniques that are giving the best translating accuracy. The four major techniques are Statistical Machine Translation, Rule-based Machine Translation, Example-based Machine Translation and Neural Machine Translation. We will see comparison and evaluation between these techniques in this particular paper with the examples and BLEU scores.

Progress in Machine Translation

After more than 70 years of evolution, great achievements have been made in machine translation. Especially in recent years, translation quality has been greatly improved with the emergence of neural machine translation (NMT). In this article, we first review the history of machine translation from rule-based machine translation to example-based machine translation and statistical machine translation. We then introduce NMT in more detail, including the basic framework and the current dominant framework, Transformer, as well as multilingual translation models to deal with the data sparseness problem. In addition, we introduce cutting-edge simultaneous translation methods that achieve a balance between translation quality and latency. We then describe various products and applications of machine translation. At the end of this article, we briefly discuss challenges and future research directions in this field.

Scaling up a Hybrid MT System: From low to full resources

This article describes a hybrid approach to machine translation (MT) that is inspired by the rule-based, statistical, example-based, and other hybrid machine translation approaches currently used or described in academic literature. It describes how the approach was implemented for language pairs using only limited monolingual resources and hardly any parallel resources (the METIS-II system), and how it is currently implemented with rich resources on both the source and target side as well as rich parallel data (the PaCo-MT system). We aim to illustrate that a similar paradigm can be used, irrespectively of the resources available, but of course with an impact on translation quality.

Machine translation using natural language processing

Machine Translation is the translation of text or speech by a computer with no human involvement. It is a popular topic in research with different methods being created, like rule-based, statistical and examplebased machine translation. Neural networks have made a leap forward to machine translation. This paper discusses the building of a deep neural network that functions as a part of end-to-end translation pipeline. The completed pipeline would accept English text as input and return the French Translation. The project has three main parts which are preprocessing, creation of models and Running the model on English Text.

Pause-Based Phrase Extraction and Effective OOV Handling for Low-Resource Machine Translation Systems

Machine translation is the core problem for several natural language processing research across the globe. However, building a translation system involving low-resource languages remains a challenge with respect to statistical machine translation (SMT). This work proposes and studies the effect of a phrase-induced hybrid machine translation system for translation from English to Tamil, under a low-resource setting. Unlike conventional hybrid MT systems, the free-word ordering feature of the target language Tamil is exploited to form a re-ordered target language model and to extend the parallel text corpus for training the SMT. In the current work, a novel rule-based phrase-extraction method, implemented using parts-of-speech (POS) and place-of-pause in both languages is proposed, which is used to pre-process the training corpus for developing the back-off phrase-induced SMT. Further, out-of-vocabulary (OOV) words are handled using speech-based transliteration and two-level thesaurus intersection techniques based on the POS tag of the OOV word. To ensure that the input with OOV words does not skip phrase-level translation in the hierarchical model, a phrase-level example-based machine translation approach is adopted to find the closest matching phrase and perform translation followed by OOV replacement. The proposed system results in a bilingual evaluation understudy score of 84.78 and a translation edit rate of 19.12. The performance of the system is compared in terms of adequacy and fluency, with existing translation systems for this specific language pair, and it is observed that the proposed system outperforms its counterparts.

Analogical-Based Translation Hypothesis Derivation with Structural Semantics for English to Malay Example-Based Machine Translation

An analogical-based approach towards meaning preservation by transferring the source language meaning structure to the target language in the recombination process of an English to Malay Example-based Machine Translation system is presented. The meaning structure is built on top of the current synchronized translation examples pair representation with the incorporation of a layer of semantic annotation at the structural level. This meaning structure provides a consistent medium allowing the derivation of translation hypothesis using analogical-based approach throughout the automated translation process. The complexity of the structural transformation in the final recombination process is relaxed with this analogical-based derivation approach. The preliminary experiment demonstrates that the English to Malay automated translation is improved using the analogical-based approach. The best evaluation score is obtained using the Bilingual Evaluation Understudy metric, showing improvement of 37.06%.

Meaning preservation in Example-based Machine Translation with structural semantics

The main tasks in Example-based Machine Translation (EBMT) comprise of source text decomposition, following with translation examples matching and selection, and finally adaptation and recombination of the target translation. As the natural language is ambiguous in nature, the preservation of source text’s meaning throughout these processes is complex and challenging. A structural semantics is introduced, as an attempt towards meaning-based approach to improve the EBMT system. The structural semantics is used to support deeper semantic similarity measurement and impose structural constraints in translation examples selection. A semantic compositional structure is derived from the structural semantics of the selected translation examples. This semantic compositional structure serves as a representation structure to preserve the consistency and integrity of the input sentence’s meaning structure throughout the recombination process. In this paper, an English to Malay EBMT system is presented to demonstrate the practical application of this structural semantics. Evaluation of the translation test results shows that the new translation framework based on the structural semantics has outperformed the previous EBMT framework.

Large aligned treebanks for syntax-based machine translation

We present a collection of parallel treebanks that have been automatically aligned on both the terminal and the non-terminal constituent level for use in syntax-based machine translation. We describe how they were constructed and applied to a syntax- and example-based machine translation system called Parse and Corpus-Based Machine Translation (PaCo-MT). For the language pair Dutch to English, we present non-terminal alignment evaluation scores for a variety of tree alignment approaches. Finally, based on the parallel treebanks created by these approaches, we evaluate the MT system itself and compare the scores with those of Moses, a current state-of-the-art statistical MT system, when trained on the same data.

Vélþýðingar á íslensku og Apertium-þýðingarkerfið

There has been rapid development in language technology and machine translation in recent decades. There are three main types of machine translation: statistical ma-chine translation, rule-based machine translation, and example-based machine translation. In this article the Apertium machine translation system is discussed in particular. While Apertium was originally designed to translate between closely related languages, it can now handle languages that are much more different and variable in structure. Anyone can participate in the development of the Apertium system since it is an open source soft ware. Thus Apertium is one of the best options available in order to research and develop a machine translation system for Icelandic. The Apertium system has an easy-to-use interface, and it translates almost instantly from Icelandic into English or Swedish. However, the system still has certain limitations as regards vocabulary and ambiguity.

Example-based Machine Translation from English to Farsi with the help of WordNet Ontology

Machine translation is a process of automatic translation from one language to another. In the present study, architecture was proposed for the system of example-based machine translation from English to Farsi with the help of WordNet ontology. This system uses a knowledge base containing a set of examples, a bilingual dictionary and WordNet. An effective factor in quality of translation is ambiguity in selecting one of the different meanings that a word can have while being translated from one language to another. In the proposed method of the present study, WorldNet is utilized in the formation phase of example set and the adaptation phase so that ambiguity in translation can be reduced and selection of the correct meaning from among different meanings will be possible.

A Pure EBMT Approach for English to Hindi Sentence Translation System

The paper focuses on Example Based Machine Translation (EBMT) system that translates sentences from English to Hindi. It uses the parallel corpus for translating sentences. Development of a machine translation (MT) system typically demands a large volume of computational resources. Requirement of computational resources (for example, rules) is much less in respect of EBMT. This makes development of EBMT systems for English to Hindi translation feasible, where availability of large-scale computational resources is still scarce. Example based machine translation relies on the database for its translation. The frequency of word occurrence is important for translation in EBMT in the following research.

NLP CHALLENGES FOR MACHINE TRANSLATION FROM ENGLISH TO INDIAN LANGUAGES

This Natural Langauge processing is carried particularly on English-Kannada/Telugu. Kannada is a language of India. The Kannada language has a classification of Dravidian, Southern, Tamil-Kannada, and Kannada. Regions Spoken: Kannada is also spoken in Karnataka, Andhra Pradesh, Tamil Nadu, and Maharashtra. Population: The total population of people who speak Kannada is 35,346,000, as of 1997. Alternate Name: Other names for Kannada are Kanarese, Canarese, Banglori, and Madrassi. Dialects: Some dialects of Kannada are Bijapur, Jeinu Kuruba, and Aine Kuruba. There are about 20 dialects and Badaga may be one. Kannada is the state language of Karnataka. About 9,000,000 people speak Kannada as a second language. The literacy rate for people who speak Kannada as a first language is about 60%, which is the same for those who speak Kannada as a second language (in India). Kannada was used in the Bible from 1831-2000. Statistical machine translation (SMT) is a machine translation paradigm where translations are generated on the basis of statistical models whose parameters are derived from the analysis of bilingual text corpora. The statistical approach contrasts with the rule-based approaches to machine translation as well as with example-based machine translation.

Malayalam To English Machine Translation:An EBMT System

Machine Translation became important in the field of communication around the world with people having their own native language. Machine translation is one of the research areas under computational linguistics. Various methods have been proposed to automate the translation process. The thesis proposes a Machine Translation system for transaction from Malayalam to English language. The translation system is based on Example Based Machine Translation (EBMT) approach. The input to the translation system is Malayalam sentence and the corresponding English sentence is generated as output. Example Based machine translation is based on the idea of reusing the already translated examples. Example based translation involves three major steps Example acquisition, Matching and Recombination. It is founded that the translation system works well for the simple sentence in Malayalam language.

Urdu to English Machine Translation using Bilingual Evaluation Understudy

Machine Translation (MT) is exigent because it involves several thorny subtasks such as intrinsic language ambiguities, linguistic complexities and diversities between source and target language. Usually MT depends upon rules that provide linguistic information. At present, the corpus based MT approaches are used that include techniques like Example Based MT (EBMT) and Statistical MT (SMT). In addition to others, both of these corpus based techniques have different frameworks in the contemporary data-driven paradigm. SMT systems generate outputs using probabilities, whereas EBMT systems translate input text by matching examples from large amount of training data. Urdu MT is in its infancy with very limited availability of required data and computational resources. In this paper, we analyzed and evaluated the main MT techniques using qualitative as well as quantitative approaches. Strengths and weaknesses of each technique have been brought to light through special focus and discussion on examples from Urdu language MT literature. We evaluated the automated machine translated outputs using Bilingual Evaluation Understudy (BLEU). The EBMT approach produced the highest accuracy of 84.21% whereas the accuracy of the online SMT system is 62.68%. We found that BLUE scores of machine translated long Urdu sentences are low in comparison with long sentences. Similarly source text containing low frequency words affect the quality of Urdu machine translation negatively. Experiments and findings section of this paper explicate our reported results in detail. The paper concludes with proposal of future directions for research in Urdu machine translation.

Chinese-Japanese Machine Translation Exploiting Chinese Characters

The Chinese and Japanese languages share Chinese characters. Since the Chinese characters in Japanese originated from ancient China, many common Chinese characters exist between these two languages. Since Chinese characters contain significant semantic information and common Chinese characters share the same meaning in the two languages, they can be quite useful in Chinese-Japanese machine translation (MT). We therefore propose a method for creating a Chinese character mapping table for Japanese, traditional Chinese, and simplified Chinese, with the aim of constructing a complete resource of common Chinese characters. Furthermore, we point out two main problems in Chinese word segmentation for Chinese-Japanese MT, namely, unknown words and word segmentation granularity, and propose an approach exploiting common Chinese characters to solve these problems. We also propose a statistical method for detecting other semantically equivalent Chinese characters other than the common ones and a method for exploiting shared Chinese characters in phrase alignment. Results of the experiments carried out on a state-of-the-art phrase-based statistical MT system and an example-based MT system show that our proposed approaches can improve MT performance significantly, thereby verifying the effectiveness of shared Chinese characters for Chinese-Japanese MT.

Word Similarity Computation Based on WordNet and HowNet

Word similarity computation is broadly used in many applications, such as information retrieval, information extraction, text categorization, word sense disambiguation and example-based machine translation and so on. The main obstacle of word similarity computation lie in that how to develop a computational algorithm that is capable of generating satisfactory results close to how human perceive. This paper proposed an approach of word similarity computation which combined WordNet and HowNet. Experiments on Chinese word pairs show that our method is closest to human similarity judgments when compared to the major state-of-art methods.

Briefly Noted

Challenges for Arabic Machine TranslationAbdelhadi Soudi1, Ali Farghaly2, Günter Neumann3, and Rabih Zbib4(editors)(1École Nationale de l'Industrie Minérale, 2Monterey Institute of International Studies, 3German Research Center for Artificial Intelligence, 4BBN Technologies)John Benjamins Publishing Company (Natural language processing series, edited by Ruslan Mitkov, volume 9), 2012, viii+157 pp; hardbound, ISBN 978-90-272-4995-1, $135.00, €90.00As the name of the book suggests, Arabic is a rather challenging language to handle in a machine translation (MT) system. The main challenges that are discussed and tackled throughout the book include Arabic complex morphology and a different syntactic structure from English.The book assembles recent work targeting Arabic-to-English and English-to-Arabic machine translation. Paradigms for MT include statistical MT (SMT) and example-based MT (EBMT). Techniques to improve the MT quality include preprocessing (Arabic segmentation, reordering) and syntactic models for SMT, and generalized matching for EBMT. The domains of research that are presented in the book are rather broad, but it lacks a unified experimental environment, rendering a comparison of the empirical results hard. One additional caveat is the partial quantitative information about Arabic translation challenges. Statistics about Arabic ambiguity and about the amount and complexity of reorderings, and a comparison to other language pairs (e.g., German–English), could be meaningful and relate well to the title of the book. Additionally, this information can justify the need for a special treatment of the Arabic language within the MT community.Finally, the strong side of the book includes exploration of less dominant research fields as English-to-Arabic MT and EBMT. A comparison of SMT and EBMT, and a discussion of advantages and disadvantages of the paradigms is also given. I would recommend the book for MT professionals as a useful starting point containing references and established work on Arabic MT.—Saab Mansour, RWTH Aachen University, Aachen, GermanySaab Mansour is a research assistant and a Ph.D. student at RWTH Aachen University since 2008. Mansour's address is Chair of Computer Science 6, RWTH Aachen University, Aachen, Germany; e-mail: mansour@cs.rwth-aachen.de.

How to Translate Unknown Words for English to Bangla Machine Translation Using Transliteration

Due to small available English-Bangla parallel corpus, Example-Based Machine Translation (EBMT) system has high probability of handling unknown words. To improve translation quality for Bangla language, we propose a novel approach for EBMT using WordNet and International-Phonetic-Alphabet(IPA)-based transliteration. Proposed system first tries to find semantically related English words from WordNet for the unknown word. From these related words, we choose the semantically closest related word whose Bangla translation exists in English-Bangla dictionary. If no Bangla translation exists, the system uses IPA-based-transliteration. If unknown word is not found in the English IPA dictionary, the system uses Akkhor transliteration mechanism. We implemented the proposed approach in EBMT, which improved the quality of good translation by 16 points.

Improving Example-Based Machine Translation with Statistical Collocation Model

基于实例的机器翻译(example-based machine translation,简称EBMT)使用预处理过的双语例句作为主要翻译资源,通过编辑与待翻译句子匹配的翻译实例来生成译文.在EBMT系统中,翻译实例选择及译文选择对系统性能影响较大.提出利用统计搭配模型来增强EBMT 系统中翻译实例选择及译文选择的能力,提高译文质量.首先,使用单语统计词对齐从单语语料中训练统计搭配模型.然后,利用该模型从3 个方面提高EBMT 的性能:(1) 利用统计搭配模型估计待翻译句子与翻译实例之间的匹配度,从而增强系统的翻译实例选择能力;(2) 通过引入候选译文与上下文之间搭配强度的估计来提高译文选择能力;(3) 使用统计搭配模型检测翻译实例中被替换词的搭配词,同时根据新的替换词及上下文对搭配词进行矫正,进一步提高EBMT系统的译文质量.为了验证所提出的方法,在基于词的EBMT 系统上评价了英汉翻译的译文质量.与基线系统相比,所提出的方法使译文的BLEU 得分提高了4.73~6.48个百分点.在半结构化的EBMT系统上进一步检验了基于统计搭配模型的译文选择方法,从实验结果来看,该方法使译文的BLEU 得分提高了1.82 个百分点.同时,人工评价结果显示,改进后的半结构化EBMT 系统的译文能够表达原文的大部分信息,并且具有较高的流利度.

Research on Example-based Machine Translation for Oracle Bone Inscriptions

Research on Example-based Machine Translation for Oracle Bone Inscriptions