Distantly Supervised Relation Extraction Based on Multi-head Self-attention and Gate Mechanism

Abstract

AbstractIn the field of distant supervision relation extraction, PCNN (piecewise convolution neural network) is normally involved to trap local features of sentences, and has achieved good results. However, the existing PCNN-based methods are unable to capture the features of long-distance interdependence in sentences and cannot distinguish the influence of the three segments of PCNN on relation classification, which probably results in missing some vital information. To address the above two issues, we come up with a new model with multi-head self-attention and gate mechanism for distant supervised relation extraction. Firstly, to capture the features of long-distance interdependence in sentences, we employ an internal multi-head self-attention in PCNN model, which can grab information in different representation subspaces. Secondly, to distinguish the influence of the three segments of the piecewise max-pooling output on the relation classification, the gate mechanism is introduced to assign different weights to the three segments and highlight important segments. After a series of experiments, it is proven that the model we presented is better than the previous approaches on each evaluation criterion.KeywordsMulti-head self-attentionGate mechanismRelation extractionDistantly supervised