Abstract

Named entity recognition (NER) is highly sensitive to sentential syntactic and semantic properties where entities may be extracted according to how they are used and placed in the running text. To model such properties, one could rely on existing resources to providing helpful knowledge to the NER task; some existing studies proved the effectiveness of doing so, and yet are limited in appropriately leveraging the knowledge such as distinguishing the important ones for particular context. In this paper, we improve NER by leveraging different types of syntactic information through attentive ensemble, which functionalizes by the proposed key-value memory networks, syntax attention, and the gate mechanism for encoding, weighting and aggregating such syntactic information, respectively. Experimental results on six English and Chinese benchmark datasets suggest the effectiveness of the proposed model and show that it outperforms previous studies on all experiment datasets.

Highlights

  • We propose a sequence labeling based neural model to enhance Named entity recognition (NER) by incorporating different types of syntactic information, which is conducted by attentive ensemble with key-value memory networks (KVMN) (Miller et al, 2016), syntax attention and the gate mechanism

  • To enhance NER with the syntactic information encoded by KVMN and combined by syntax attention, we propose a gate mechanism (GM) to incorporate it to the backbone NER model, where we expect such mechanism could dynamically weight and decide how to leverage the syntactic information in labeling named entities (NEs)

  • To explore how different syntactic information helps NER, we run the baselines without syntactic information and the ones with each type of syntactic information through KVMN.15

Read more

Summary

Introduction

Named entity recognition (NER) is one of the most important and fundamental tasks in natural language processing (NLP), which identifies named entities (NEs), such as locations, organizations, person names, etc., in running texts, and plays an important role in downstream NLP applications including question answering (Pang et al, 2019), semantic parsing (Dong and Lapata, 2018) and entity linking (Martins et al, 2019), etc. We propose a sequence labeling based neural model to enhance NER by incorporating different types of syntactic information, which is conducted by attentive ensemble with key-value memory networks (KVMN) (Miller et al, 2016), syntax attention and the gate mechanism. Important syntactic information is highlighted and emphasized during labeling NEs. In addition, to further improve NER performance, we try different types of pre-trained word embeddings, which is demonstrated to be effective in previous studies (Akbik et al, 2018; Jie and Lu, 2019; Liu et al, 2019b; Yan et al, 2019). Experimental results on all datasets suggest the effectiveness of our approach to enhance NER through syntactic information, where state-of-theart results are achieved on all datasets

The Proposed Model
Syntactic Information Extraction
KVMN for Syntactic Information
The Syntax Attention
The Gate Mechanism
Encoding and Decoding for NER
Datasets
Implementation
Effect of Key-Value Memory Networks
Effect of Syntax Attention
Effect of the Gate Mechanism
Comparison with Previous Studies
Effect of Different Word Embeddings
Case Study
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call