Multi-Level Attention Based Coreference Resolution With Gated Recurrent Unit and Convolutional Neural Networks

Abstract

Aiming at the diversity of the entity mentions in the field of animal husbandry in Tibet, the reference resolution model Att-GRU-CNN based on multi-level attention mechanism is proposed. The model uses GRU network for global semantic feature learning and knowledge memory, and uses CNN network to further extract local high-level semantic features. A word-level attention layer is added on the GRU hidden layer, the feature vector of entity mention does dot product operation with the feature vector of each word, and the result is normalized and used as the weight value distribution of the words, thus, the prior knowledge that the mention is the most important in the context is given to the network; A sentence-level attention layer is added on the CNN convolution layer, the convolutional layer output and entity name library matrix do association operation, and the result is used as the weight value distribution of the sentence, so as to strengthen the relation between the sentence where the mention is located and the entity scientific name. At the same time, the data enhancement technology is used to improve the generalization ability of the model. Finally, the ablation experiment on Tibetan animal husbandry dataset verified the effectiveness of each component of the model and the superposition effect after combination; the performance comparison experiment is carried out on public datasets MUC, B3 and CEAF <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\varphi _{4}$ </tex-math></inline-formula> . The experimental results show that this model has a significant improvement over other models in the accuracy rate, recall rate and F1 score.