Phrase Pairs Research Articles

Phrase-boundary model for statistical machine translation

This paper proposes a new probabilistic synchronous context-free grammar model for statistical machine translation. The model labels nonterminals with classes of boundary words on the target side of aligned phrase pairs. Labeling of the rules is performed with coarse grained and fine grained nonterminals using POS tags and word clusters trained on the target language corpus. Considering the large size of the proposed model due to the diversity of nonterminals, we have also proposed a novel approach for filtered rule extraction based on the alignment pattern of phrase pairs. Using limited patterns of rules, the extraction of hierarchical rules gets restricted from phrase pairs that are decomposable to two aligned subphrases. The proposed filtered rule extraction decreases the model size and the decoding time considerably with no significant impact on the translation quality. Using BLEU as a metric in our experiments, the proposed model achieved a notable improvement rate over the state-of-the-art hierarchical phrase-based model in the translation from Persian, French and Spanish to English language. This is applicable for all languages, even under-resourced ones having no linguistic tools.

Labeling hierarchical phrase-based models without linguistic resources

Long-range word order differences are a well-known problem for machine translation. Unlike the standard phrase-based models which work with sequential and local phrase reordering, the hierarchical phrase-based model (Hiero) embeds the reordering of phrases within pairs of lexicalized context-free rules. This allows the model to handle long range reordering recursively. However, the Hiero grammar works with a single nonterminal label, which means that the rules are combined together into derivations independently and without reference to context outside the rules themselves. Follow-up work explored remedies involving nonterminal labels obtained from monolingual parsers and taggers. As of yet, no labeling mechanisms exist for the many languages for which there are no good quality parsers or taggers. In this paper we contribute a novel approach for acquiring reordering labels for Hiero grammars directly from the word-aligned parallel training corpus, without use of any taggers or parsers. The new labels represent types of alignment patterns in which a phrase pair is embedded within larger phrase pairs. In order to obtain alignment patterns that generalize well, we propose to decompose word alignments into trees over phrase pairs. Beside this labeling approach, we contribute coarse and sparse features for learning soft, weighted label-substitution as opposed to standard substitution. We report extensive experiments comparing our model to two baselines: Hiero and the known syntax augmented machine translation (SAMT) variant, which labels Hiero rules with nonterminals extracted from monolingual syntactic parses. We also test a simplified labeling scheme based on inversion transduction grammar (ITG). For the Chinese---English task we obtain performance improvement up to 1 BLEU point, whereas for the German---English task, where morphology is an issue, a minor (but statistically significant) improvement of 0.2 BLEU points is reported over SAMT. While ITG labeling does give a performance improvement, it remains sometimes suboptimal relative to our proposed labeling scheme.

From Paraphrase Database to Compositional Paraphrase Model and Back

The Paraphrase Database (PPDB; Ganitkevitch et al., 2013) is an extensive semantic resource, consisting of a list of phrase pairs with (heuristic) confidence estimates. However, it is still unclear how it can best be used, due to the heuristic nature of the confidences and its necessarily incomplete coverage. We propose models to leverage the phrase pairs from the PPDB to build parametric paraphrase models that score paraphrase pairs more accurately than the PPDB’s internal scores while simultaneously improving its coverage. They allow for learning phrase embeddings as well as improved word embeddings. Moreover, we introduce two new, manually annotated datasets to evaluate short-phrase paraphrasing models. Using our paraphrase model trained using PPDB, we achieve state-of-the-art results on standard word and bigram similarity tasks and beat strong baselines on our new short phrase paraphrase tasks.

Extraction of Potentially Useful Phrase Pairs for Statistical Machine Translation

Extraction of Potentially Useful Phrase Pairs for Statistical Machine Translation

Learning relational facts from the web: A tolerance rough set approach

A key issue when mining web information is the labeling problem: data are abundant on the web but is unlabeled. In this paper, we address this problem by proposing (i) a granular model that structures categorical noun phrase instances as well as semantically related noun phrase pairs from a given corpus representing unstructured web pages with a tolerance form of rough sets, (ii) a semi-supervised Tolerant Pattern Learning (TPL) algorithm that labels categorical instances as well as relations. This work is an extension of the TPL algorithm presented in our earlier paper. Our model treats noun phrases, which are described as sets of their co-occurring contextual patterns. We use the ontological information from the Never Ending Language Learner (Nell) system. We compared the performance of our algorithm with Coupled Bayesian Sets (CBS) and Coupled Pattern Learner (CPL) algorithms for categorical and relational extractions, respectively. Experimental results suggest that TPL can achieve comparable performance with CBS and CPL in terms of precision.

Online adaptation to post-edits for phrase-based statistical machine translation

Recent research has shown that accuracy and speed of human translators can benefit from post-editing output of machine translation systems, with larger benefits for higher quality output. We present an efficient online learning framework for adapting all modules of a phrase-based statistical machine translation system to post-edited translations. We use a constrained search technique to extract new phrase-translations from post-edits without the need of re-alignments, and to extract phrase pair features for discriminative training without the need for surrogate references. In addition, a cache-based language model is built on $$n$$n-grams extracted from post-edits. We present experimental results in a simulated post-editing scenario and on field-test data. Each individual module substantially improves translation quality. The modules can be implemented efficiently and allow for a straightforward stacking, yielding significant additive improvements on several translation directions and domains.

Topic-aware pivot language approach for statisticalmachine translation

The pivot language approach for statistical machine translation (SMT) is a good method to break the resource bottleneck for certain language pairs. However, in the implementation of conventional approaches, pivot-side context information is far from fully utilized, resulting in erroneous estimations of translation probabilities. In this study, we propose two topic-aware pivot language approaches to use different levels of pivot-side context. The first method takes advantage of document-level context by assuming that the bridged phrase pairs should be similar in the document-level topic distributions. The second method focuses on the effect of local context. Central to this approach are that the phrase sense can be reflected by local context in the form of probabilistic topics, and that bridged phrase pairs should be compatible in the latent sense distributions. Then, we build an interpolated model bringing the above methods together to further enhance the system performance. Experimental results on French-Spanish and French-German translations using English as the pivot language demonstrate the effectiveness of topic-based context in pivot-based SMT.

Improvement of the Results of Statistical Machine Translation System using Anusaaraka

This paper describes an efficient experimental approach for the improvement of translation quality of phrase based statistical machine translation system by utilizing the insights of the rule based machine translation. As the most primitive step it is believed that appending large and accurately designed linguistic resources such as multiword bilingual dictionaries to the existing training corpus contributes a lot towards the enhancement of phrase alignment quality and phrase coverage of the Statistical Machine Translation (SMT) system. Further improvement in translation coverage can be achieved by improving the dictionary by introducing morphsyntactic word forms of the foreign language words instead of simple root word forms, and its corresponding translations in native language. As in real time testing scenario, the test corpus may possess different morphological extensions of the root word which is not covered by standard dictionaries. As a matter of fact addition of such dictionaries to the corpus enriches it and provides a solution to the improper translations previously generated due to occurrences of morph-syntactic extensions instead of the root word form. As the proposed approach towards further improvement, the intelligence of Anusaaraka and huge computational ability of SMT is integrated to achieve better translations. Anusaaraka is a machine translation system based on Panini’s Astadhyayi grammatical rules and an expert when the English-Hindi phrase alignment is concerned. It does it by comparing its output translation with the accurate manual translation and extracting out the best possible option. The bi-lingual phrase pairs thus obtained are highly accurate and when appended to the training corpus of statistical machine translation system results as better phrase alignment structure, hence better translation quality.

A Phrase Table Filtering Model Based on Binary Classification for Uyghur-Chinese Machine Translation

In statistical machine translation, large amount of unreasonable phrase pairs in a phrase table can affect the decoding efficiency and the overall translation performance, especially in Uyghur-Chinese machine translation. In this paper, we present a novel phrase table filtering model based on binary classification, which consider differences between Uyghur and Chinese, and draw lessons from binary classification in machine learning. In our model, four features are considered: 1) Difference in length between source and target phrase; 2) Proportion of translated words in phrase pairs; 3) Proportion of symbol words; 4) Average number of co-occurrence words in training corpus. We use this model to generate a filtered phrase table. Experimental results show that this new filtering model can improve the performance and efficiency of our current Uygur-Chinese machine translation system.

A relationship: word alignment, phrase table, and translation quality.

In the last years, researchers conducted several studies to evaluate the machine translation quality based on the relationship between word alignments and phrase table. However, existing methods usually employ ad-hoc heuristics without theoretical support. So far, there is no discussion from the aspect of providing a formula to describe the relationship among word alignments, phrase table, and machine translation performance. In this paper, on one hand, we focus on formulating such a relationship for estimating the size of extracted phrase pairs given one or more word alignment points. On the other hand, a corpus-motivated pruning technique is proposed to prune the default large phrase table. Experiment proves that the deduced formula is feasible, which not only can be used to predict the size of the phrase table, but also can be a valuable reference for investigating the relationship between the translation performance and phrase tables based on different links of word alignment. The corpus-motivated pruning results show that nearly 98% of phrases can be reduced without any significant loss in translation quality.

Maximum-entropy word alignment and posterior-based phrase extraction for machine translation

One of the fundamental assumptions in statistical machine translation (SMT) is that the correspondence between a sentence and its translation can be explained in terms of an alignment between their words. Such alignment information is typically not observed in the parallel corpora used to build the phrase table of an SMT system. Therefore, it is customary to estimate a probabilistic model of the assumed hidden word alignment, which is then used to extract bilingual phrase pairs. In standard extraction heuristics, the alignment model is under-exploited as the only information used from the posterior distribution is the Viterbi best alignment. This is due to the high computational complexity of the IBM models, which are the de facto standard for computing these alignments. Note that these models have other limitations, including their asymmetry and their inability to integrate rich, feature-based, descriptions. We argue that refining the word alignment model in a discriminative maximum-entropy framework substantially improves the alignment quality. We also show that these improved alignments combined with efficient and accurate computation of the link posterior distributions can also improve the overall translation performance, especially when applying posterior-based extraction methods.

Clarification Question Generation for Speech Recognition Error Recovery Using Monolingual SMT

Clarification dialogue is an efficient and direct way of handling speech recognition errors in speech interface applications. In this paper we present a new approach to Clarification Question (CQ) generation. Monolingual phrase-based SMT (PB-SMT) framework is introduced to generate robust and flexible CQs. A parallel corpus from simulated error to manually annotated CQ is established and used for training the model. A new type of generalized phrase pair is expanded from conventional translation phrase table. Combining both generalized and conventional phrase pairs, a two-step decoding process is carried out to generate CQs. Both manually and automatic metrics are used to evaluate the quality of generated CQs. Experimental results show that our method can effectively generate reasonable CQs form miss-recognized utterances, and generated CQs can be used to prompt a clarification dialogue for error handling.

An Investigation of the Sampling-Based Alignment Method and its Contributions

By investigating the distribution of phrase pairs in phrase translation tables, the work in this paper describes an approach to increase the number of n-gram alignments in phrase translation tables output by a sampling-based alignment method. This approach consists in enforcing the alignment of n-grams in distinct translation subtables so as to increase the number of n-grams. Standard normal distribution is used to allot alignment time among translation subtables, which results in adjustment of the distribution of n- grams. This leads to better evaluation results on statistical machine translation tasks than the original sampling-based alignment approach. Furthermore, the translation quality obtained by merging phrase translation tables computed from the sampling-based alignment method and from MGIZA++ is examined.

Decorated Phrase Model and Syntax-Based Reordering Model for Statistical Machine Translation

In the past few years, much attention has been paid on extending phrase-based statistical machine translation with syntactic structures. In this paper, we introduce a novel phrase model, in which treebank tags are employed to decorate the bilingual phrase pairs. We use tag sequences, instead of phrase pairs, to train the lexicalized reordering model. Since the number of treebank tags is much smaller than the number of words, the tag sequence based reordering model is smaller and more accurate than the phrase based reordering model. Experiments were carried out on three types of models: the phrase model, the POS tag encapsulated phrase (PTEP) model and the syntactic tag encapsulated phrase (STEP) model. The STEP model obtained higher BLEU-4 score than other models on NIST MT tasks.

Eppex: Epochal Phrase Table Extraction for Statistical Machine Translation

eppex: Epochal Phrase Table Extraction for Statistical Machine Translation We present a tool that extracts phrase pairs from a word-aligned parallel corpus and filters them on the fly based on a user-defined frequency threshold. The bulk of phrase pairs to be scored is much reduced, making the whole phrase table construction process faster with no significant harm to the ultimate phrase table quality as measured by BLEU. Technically, our tool is an alternative to the extract component of the phrase-extract toolkit bundled with Moses SMT software and covers some of the functionality of sigfilter.

Improving Phrasebased Statistical Myanmar to English Machine Translation with Morphological Analysis

paper presents Myanmar phrases translation model with morphology analysis. The system is based on statistical approach. In statistical machine translation, large amount of information is needed to guide the translation process. When small amount of training data is available, morphological analysis is needed especially for morphology rich language. Myanmar language is inflected language and there are very few creations and researches of corpora in Myanmar, comparing to other language such as English, French, and Czech etc. Therefore, Myanmar phrases translation model is based on syntactic structure and morphology of Myanmar language. Bayes rule is also used to reformulate the translation probability of phrase pairs. Experiment results showed that proposed system can improve translation quality by applying morphological analysis on Myanmar language.

Roles of syntax information in directing song development in white-crowned sparrows (zonotrichia leucophrys).

Syntactical cues play an important role in song learning in songbirds. White-crowned sparrows (Zonotrichia leucophrys), whose song typically consists of four to five different phrases, fail to construct normal songs if exposed to all phrase types presented singly (Plamondon, Goller, & Rose, 2008; Soha & Marler 2001b). The specific role of acquired syntax information in guiding ontogenetic trajectories of syntax, however, and the respective contributions of instructive and selective processes to syntax ontogeny remain unknown. We tutored white-crowned sparrows with syntax information ranging from acoustic isolation to full song. Manipulation of tutor syntax influenced developmental trajectories of syntax assembly, suggesting that instructive processes contribute to syntax ontogeny. Early in development, birds tutored with full song or phrase pairs preferentially produced phrase pairings matching tutor syntax. Birds tutored with single phrases showed decreased diversity of pairwise syntactical combinations immediately after tutoring compared with other tutor groups, further illustrating the role of instructive processes. Overproduction of song material was also observed, suggesting that selective forces play a role in syntax development as well. Finally, consistent with the notion that innate influences guide syntax ontogeny, birds from all groups exhibited many similarities in trajectories of syntax assembly.

Automatically generated parallel treebanks and their exploitability in machine translation

Given much recent discussion and the shift in focus of the field, it is becoming apparent that the incorporation of syntax is the way forward for improvements to the current state-of-the-art in machine translation (MT). Parallel treebanks are a relatively recent innovation and appear to be ideal candidates for MT training material. However, until recently there has been no other means to build them than by hand. In this paper, we describe how we make use of new tools to automatically build a large parallel treebank and extract a set of linguistically-motivated phrase pairs from it. We show that adding these phrase pairs to the translation model of a baseline phrase-based statistical MT (PB-SMT) system leads to significant improvements in translation quality. Following this, we describe experiments in which we exploit the information encoded in the parallel treebank in other areas of the PB-SMT framework, while investigating the conditions under which the incorporation of parallel treebank data performs optimally. Finally, we discuss the possibility of exploiting automatically-generated parallel treebanks further in syntax-aware paradigms of MT.

Tutor model syntax influences the syntactical and phonological structure of crystallized songs of white-crowned sparrows

Songbirds undergo a sensitive period in which exposure to adult song is necessary for normal vocal development. During this period, a template is formed and later used as a model for comparison via auditory feedback. In montane white-crowned sparrows, Zonotrichia leucophrys oriantha, whose song typically consists of four to five distinct segments (phrases), exposure to syntactically overlapping phrase pairs during the sensitive period provides the minimum sufficient information for song assembly; exposure to isolated phrases is insufficient. This suggests that a template representing phrase-pair information is sufficient for guiding normal song production. However, birds may also store and use the additional information they receive when tutored with full song. In the present study, we tutored white-crowned sparrows with a range of phrase sequence information. We found that sparrows tutored with varying levels of syntax information differed in the syntactical (phrase sequence) and phonological (phrase structure) characteristics of their final songs. Sparrows tutored with isolated phrases produced songs that were syntactically and, surprisingly, phonologically deficient. The crystallized songs of birds tutored with full song generally consisted of five phrase types, whereas those of birds tutored with phrase pairs generally had four phrase types. In addition, birds tutored with phrase pairs combined phrases from different tutors in their final songs, while birds tutored with full song did not. These results suggest that the additional syntactical information present in full song is stored and later used. Also, the ability to accurately reproduce phrase structure is linked to the availability of syntax information during the sensitive period.

Syntactically Lexicalized Phrase-Based SMT

Until quite recently, extending phrase-based statistical machine translation (PBSMT) with syntactic knowledge caused system performance to deteriorate. The most recent successful enrichments of PBSMT with hierarchical structure either employ nonlinguistically motivated syntax for capturing hierarchical reordering phenomena, or extend the phrase translation table with redundantly ambiguous syntactic structures over phrase pairs. In this paper, we present an extended, harmonized account of our previous work which showed that incorporating linguistically motivated lexical syntactic descriptions, called <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">supertags</i> , can yield significantly better PBSMT systems at insignificant extra computational cost. We describe a novel PBSMT model that integrates supertags into the target language model and the target side of the translation model. Two kinds of supertags are employed: those from lexicalized tree-adjoining grammar and combinatory categorial grammar. Despite the differences between the two sets of supertags, they give similar improvements. In addition to integrating the Markov supertagging approach in PBSMT, we explore the utility of a new surface grammaticality measure based on combinatory operators. We perform various experiments on the Arabic-to-English NIST 2005 test set addressing the issues of sparseness, scalability, and the utility of system subcomponents. We show that even when the parallel training data grows very large, the supertagged system retains a relatively stable absolute performance advantage over the unadorned PBSMT system. Arguably, this hints at a performance gap that cannot be bridged by acquiring more phrase pairs. Our best result shows a relative improvement of 6.1% over a state-of-the-art PBSMT model, which compares favorably with the leading systems on the NIST 2005 task. We also demonstrate that the advantages of a supertag-based system carry over to German-English, where improvements of up to 8.9% relative to the baseline system are observed.