Tag-Enhanced Dynamic Compositional Neural Network over arbitrary tree structure for sentence representation

Abstract

Learning the distributed representation of a sentence is a fundamental operation for a variety of natural language processing tasks, such as text classification, machine translation, and text semantic matching. Tree-structured dynamic compositional networks have achieved promising performance in sentence representation due to its ability in capturing the richness of compositionality. However, existing dynamic compositional networks are mostly based on binarized constituency trees which cannot represent the inherent structural information of sentences effectively. Moreover, syntactic tag information, which is demonstrated to be useful in sentence representation, has been rarely exploited in existing dynamic compositional models. In this paper, a novel LSTM structure, ARTree-LSTM, is proposed to handle general constituency trees in which each non-leaf node can have any number of child nodes. Based on ARTree-LSTM, a novel network model, Tag-Enhanced Dynamic Compositional Neural Network (TE-DCNN), is proposed for sentence representation learning, which contains two ARTree-LSTMs, i.e. tag-level ARTree-LSTM and word-level ARTree-LSTM. The tag-level ARTree-LSTM guides the word-level ARTree-LSTM in conducting dynamic composition. Extensive experiments demonstrate that the proposed TE-DCNN achieves state-of-the-art performance on text classification and text semantic matching tasks.