Center-shared sliding ensemble of neural networks for syntax analysis of natural language

Abstract

Fixing input sites in ensembles restrict learning movable and distant patterns.The new ensemble changes the sites across component classifiers sharing one position.The ensemble improves accuracy in learning movable patterns and its lower bound.The neural network framework with the ensemble improves accuracy in syntax analysis.The framework is accurate as state-of-the-art methods in Spanish dependency parsing. In this paper, we introduce a new ensemble method specialized to sequential labeling for syntax analysis and propose a neural network framework adopting the ensemble for dependency parsing of natural sentences. The ensemble method assigns sliding input sites to component classifiers which commonly include the position of the label to predict. The method improves labeling accuracy compared to simple ensemble with weighted voting if critical input features have flexible and long distance from the position to predict over sentences. We show the impact of the ensemble through theoretical estimation of its lower bound accuracy and through empirical analysis in a toy problem varying the strength of movability of critical input features. We apply the proposed neural network framework to the two phases of dependency parsing: dependency and relation tagging. Additionally, we newly define the dependency tagging problem using relative dependency and provide a post-processing method to build correct parse trees. In the practical dependency parsing of Spanish IULA corpus, applying the ensemble instead of the simple weighted voting significantly improves accuracy by 0.09% in relation tagging and by 0.06% to 1.59% with respect to the comparison settings in dependency tagging. The framework shows at least 0.28% improvement in the unlabeled attachment score and 0.14% in the labeled attachment score compared to state-of-the-art dependency parsers.