Distant Supervised Relation Extraction based on Dense Neural Network with 2-Level Attention Mechanism

Abstract

Dense connected convolutional neural network (Dense Net) is a new architecture of deep convolutional neural network, which ensures the maximum information transmission between network layers by establishing the connection relationship between different layers. In the task of text remote supervised relation extraction, aiming at the limitation of existing neural network methods using shallow network to extract features, a deep convolution neural network model with dense connection based on attention mechanism is designed. In this model, the dense connection module and maximum pooling layer composed of 5 layer convolutional neural network are used as the sentence encoder. By combining the lexical, syntactic and semantic features of different levels, it helps the network to learn the features, so as to obtain more abundant semantic information of the input sentence. At the same time, it reduces the gradient vanishing phenomenon of deep neural network, and makes the network more capable of representing natural language Strong. Secondly, the word level attention is introduced to calculate the relevance between two entities and context words, so as to fully capture the semantic information of the entity context in the sentence; then the sentence level attention is constructed on multiple instances to reduce the problem of label error labeling; finally, the dependency inclusion relationship between different relationships is automatically learned through the attention of the relationship level. The average accuracy of the model on the NYT freebase dataset is 83.5%. Experimental results show that the model can effectively use features and improve the accuracy of remote supervised relationship extraction.