Lexicalized Neural Unsupervised Dependency Parsing

Abstract

Unsupervised dependency parsing is the task of learning a dependency grammar from a set of unannotated sentences. This paper describes a neural network (NN) based probabilistic model, Neural Dependency Model with Valence (NDMV), that combines the dependency parsing with the rich nonlinear featurization of NN approaches. NDMV follows the traditional DMV model and computes the probability of a grammar rule via a feed-forward NN. A strength of this proposed approach is the ability to learn the underlying features of input tokens. In our experiments, this capability leads to gains in both NDMV and its extension version, Lexicalized Neural Dependency Model with Valence (L-NDMV): NDMV achieves better performance on WSJ and datasets of eight additional languages in comparison with previous approaches in the basic setting; L-NDMV achieves a result that is competitive with the current state-of-the-art.11Part of this paper been presented in a conference: Jiang et al. 2016 [1] and Han et al. 2017 [2].