Synchronous Tree Substitution Grammar Research Articles

Compositions of Tree-to-Tree Statistical Machine Translation Models

The well-known synchronous context-free grammars (SCFGs) and synchronous tree-substitution grammars (STSGs), both of which are used as tree-to-tree translation models in statistical machine translation are investigated. Their composition hierarchies are established in both the unweighted as well as the weighted case. More precisely, it is shown that SCFGs are closed under composition in both cases and that there is a close connection between compositions of STSGs and compositions of certain tree transducers. With the help of the close ties, the composition closure of STSGs is identified in both cases as well. The results for the weighted case utilize a new lifting technique that might prove useful also in similar setups.

Composition Closure of Linear Extended Top-down Tree Transducers

Linear extended top-down tree transducers (or synchronous tree-substitution grammars) are popular formal models of tree transformations that are extensively used in syntax-based statistical machine translation. The expressive power of compositions of such transducers with and without regular look-ahead is investigated. In particular, the restrictions of ź-freeness, strictness, and nondeletion are considered. The composition hierarchy turns out to be finite for all ź-free (all rules consume input) variants of these transducers except for the nondeleting ź-free transducers. The least number of transducers needed for the full expressive power of arbitrary compositions is presented. In all remaining cases (incl. the nondeleting ź-free transducers) the composition hierarchy does not collapse.

The Power of Weighted Regularity-Preserving Multi Bottom-Up Tree Transducers

The expressive power of regularity-preserving [Formula: see text]-free weighted linear multi bottom-up tree transducers is investigated. These models have very attractive theoretical and algorithmic properties, but (especially in the weighted setting) their expressive power is not well understood. Despite the regularity-preserving restriction, their power still exceeds that of composition chains of [Formula: see text]-free weighted linear extended top-down tree transducers with regular look-ahead. The latter devices are a natural super-class of weighted synchronous tree substitution grammars, which are commonly used in syntax-based statistical machine translation. In particular, the linguistically motivated discontinuous transformation of topicalization can be modeled by such multi bottom-up tree transducers, whereas the mentioned composition chains cannot implement it. On the negative side, the inverse of topicalization cannot be implemented by any such multi bottom-up tree transducer, which confirms their bottom-up nature (and non-closure under inverses). An interesting, promising, and widely applicable proof technique is used to prove these statements.

Bimorphisms and synchronous grammars

We tend to think of the study of language as proceeding by characterizing the strings and structures of a language, and we think of natural language processing as using those structures to build systems of utility in manipulating the language. But many language-related problems are more fruitfully viewed as requiring the specification of a relation between two languages, rather than the specification of a single language. We provide a synthesis and extension of work that unifies two approaches to such language relations: the automata-theoretic approach based on tree transducers that transform trees to their counterparts in the relation, and the grammatical approach based on synchronous grammars that derive pairs of trees in the relation. In particular, we characterize synchronous tree-substitution grammars and synchronous tree-adjoining grammars in terms of bimorphisms, which have previously been used to characterize tree transducers. In the process, we provide new approaches to formalizing the various concepts: a metanotation for describing varieties of tree automata and transducers in equational terms; a rigorous formalization of tree-adjoining and tree-substitution grammars and their synchronous counterparts, using trees over ranked alphabets; and generalizations of tree-adjoining grammar allowing multiple adjunction.

Unsupervised Tree Induction for Tree-based Translation

In current research, most tree-based translation models are built directly from parse trees. In this study, we go in another direction and build a translation model with an unsupervised tree structure derived from a novel non-parametric Bayesian model. In the model, we utilize synchronous tree substitution grammars (STSG) to capture the bilingual mapping between language pairs. To train the model efficiently, we develop a Gibbs sampler with three novel Gibbs operators. The sampler is capable of exploring the infinite space of tree structures by performing local changes on the tree nodes. Experimental results show that the string-to-tree translation system using our Bayesian tree structures significantly outperforms the strong baseline string-to-tree system using parse trees.

Syntax-Based Translation With Bilingually Lexicalized Synchronous Tree Substitution Grammars

Syntax-based models can significantly improve the translation performance due to their grammatical modeling on one or both language side(s). However, the translation rules such as the non-lexical rule “ VP→(x <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> x <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,VP:x <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> PP:x <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> )” in string-to-tree models do not consider any lexicalized information on the source or target side. The rule is so generalized that any subtree rooted at VP can substitute for the nonterminal VP:x <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> . Because rules containing nonterminals are frequently used when generating the target-side tree structures, there is a risk that rules of this type will potentially be severely misused in decoding due to a lack of lexicalization guidance. In this article, inspired by lexicalized PCFG, which is widely used in monolingual parsing, we propose to upgrade the STSG (synchronous tree substitution grammars)-based syntax translation model with bilingually lexicalized STSG. Using the string-to-tree translation model as a case study, we present generative and discriminative models to integrate lexicalized STSG into the translation model. Both small- and large-scale experiments on Chinese-to-English translation demonstrate that the proposed lexicalized STSG can provide superior rule selection in decoding and substantially improve the translation quality.

Language Modeling for Syntax-Based Machine Translation Using Tree Substitution Grammars

The poor grammatical output of Machine Translation (MT) systems appeals syntax-based approaches within language modeling. However, previous studies showed that syntax-based language modeling using (Context-Free) Treebank Grammars was not very helpful in improving BLEU scores for Chinese-English machine translation. In this article we further study this issue in the context of Chinese-English syntax-based Statistical Machine Translation (SMT) where Synchronous Tree Substitution Grammars (STSGs) are utilized to model the translation process. In particular, we develop a Tree Substitution Grammar-based language model for syntax-based MT, and present three methods to efficiently integrate the proposed language model into MT decoding. In addition, we design a simple and effective method to adapt syntax-based language models for MT tasks. We demonstrate that the proposed methods are able to benefit a state-of-the-art syntax-based MT system. On the NIST Chinese-English MT evaluation corpora, we finally achieve an improvement of 0.6 BLEU points over the baseline.

Sentence Compression as Tree Transduction

This paper presents a tree-to-tree transduction method for sentence compression. Our model is based on synchronous tree substitution grammar, a formalism that allows local distortion of the tree topology and can thus naturally capture structural mismatches. We describe an algorithm for decoding in this framework and show how the model can be trained discriminatively within a large margin framework. Experimental results on sentence compression bring significant improvements over a state-of-the-art model.

Sentence compression as tree transduction

This paper presents a tree-to-tree transduction method for sentence compression. Our model is based on synchronous tree substitution grammar, a formalism that allows local distortion of the tree to...