Multilingual Joint Parsing of Syntactic and Semantic Dependencies with a Latent Variable Model

James Henderson,Ivan Titov,Gabriele Musillo,Paola Merlo

doi:10.1162/coli_a_00158

Abstract

Current investigations in data-driven models of parsing have shifted from purely syntactic analysis to richer semantic representations, showing that the successful recovery of the meaning of text requires structured analyses of both its grammar and its semantics. In this article, we report on a joint generative history-based model to predict the most likely derivation of a dependency parser for both syntactic and semantic dependencies, in multiple languages. Because these two dependency structures are not isomorphic, we propose a weak synchronization at the level of meaningful subsequences of the two derivations. These synchronized subsequences encompass decisions about the left side of each individual word. We also propose novel derivations for semantic dependency structures, which are appropriate for the relatively unconstrained nature of these graphs. To train a joint model of these synchronized derivations, we make use of a latent variable model of parsing, the Incremental Sigmoid Belief Network (ISBN) architecture. This architecture induces latent feature representations of the derivations, which are used to discover correlations both within and between the two derivations, providing the first application of ISBNs to a multi-task learning problem. This joint model achieves competitive performance on both syntactic and semantic dependency parsing for several languages. Because of the general nature of the approach, this extension of the ISBN architecture to weakly synchronized syntactic-semantic derivations is also an exemplification of its applicability to other problems where two independent, but related, representations are being learned.

Highlights

IntroductionSuccess in statistical syntactic parsing based on supervised techniques trained on a large corpus of syntactic trees—both constituency-based (Collins 1999; Charniak 2000; Henderson 2003) and dependency-based (McDonald 2006; Nivre 2006; Bohnet and Nivre 2012; Hatori et al 2012)—has paved the way to applying statistical approaches to the more ambitious goals of recovering semantic representations, such as the logical form of a sentence (Ge and Mooney 2005; Wong and Mooney 2007; Zettlemoyer and Collins 2007; Ge and Mooney 2009; Kwiatkowski et al 2011) or learning the propositional argument-structure of its main predicates (Miller et al 2000; Gildea and Jurafsky 2002; Carreras and Marquez 2005; Marquez et al 2008; Li, Zhou, and Ng 2010)
The proposed joint model achieves competitive performance on both syntactic and semantic dependency parsing for several languages
Our experiments demonstrate the benefit of joint learning of syntax and semantics

Summary

Introduction

Success in statistical syntactic parsing based on supervised techniques trained on a large corpus of syntactic trees—both constituency-based (Collins 1999; Charniak 2000; Henderson 2003) and dependency-based (McDonald 2006; Nivre 2006; Bohnet and Nivre 2012; Hatori et al 2012)—has paved the way to applying statistical approaches to the more ambitious goals of recovering semantic representations, such as the logical form of a sentence (Ge and Mooney 2005; Wong and Mooney 2007; Zettlemoyer and Collins 2007; Ge and Mooney 2009; Kwiatkowski et al 2011) or learning the propositional argument-structure of its main predicates (Miller et al 2000; Gildea and Jurafsky 2002; Carreras and Marquez 2005; Marquez et al 2008; Li, Zhou, and Ng 2010). The recovery of the full meaning of text requires structured analyses of both its grammar and its semantics These two forms of linguistic knowledge are usually thought to be at least partly independent, as demonstrated by speakers’ ability to understand the meaning of ungrammatical text or speech and to assign grammatical categories and structures to unknown words and nonsense sentences. These two levels of representation of language, are closely correlated. From a linguistic point of view, the assumption that syntactic distributions will be predictive of semantic role assignments is based on linking theory (Levin 1986). Linking theory has been confirmed statistically (Merlo and Stevenson 2001)

Methods

Results

Conclusion