Enhancing structure modeling for relation extraction with fine-grained gating and co-attention

Abstract

Relation extraction is a critical natural language processing task. Existing dependency-based models captured long-range syntactic relations, but they usually cannot fully exploit information from sentences. They often used hand-crafted rules to prune redundant edges from dependency trees, but suffer from the imbalance of including and removing contents. When incorporating sequence models, they usually ignored the semantic and syntactic interactions between words. In this paper, we propose to automatically learn relational dependency structures with a fine-grained gating strategy. We decompose the dependency tree into differently informative parts and apply different gating methods to each part. To further capture the word-level interactions, we propose to apply the co-attention mechanism to combine structure and sequence models. We apply a neural network to learn the affinity matrix and derive mutual attention weights between semantic and syntactic representations. We conduct experiments on two benchmark datasets and the results indicate the effectiveness of our method.