Named Entity Recognition Networks Based on Syntactically Constrained Attention

Abstract

The task of named entity recognition can be transformed into a machine reading comprehension task by associating the query and its context, which contains entity information, with the encoding layer. In this process, the model learns a priori knowledge about the entity, from the query, to achieve good results. However, as the length of the context and query increases, the model struggles with an increasing number of less relevant words, which can distract it from the task. Although attention mechanisms can help the model understand contextual semantic relations, without explicit constraint information, attention may be allocated to less task-relevant words, leading to a bias in the model’s understanding of the context. To address this problem, we propose a new model, the syntactic constraint-based dual-context aggregation network, which uses syntactic information to guide query and context modeling. By incorporating syntactic constraint information into the attention mechanism, the model can better determine the relevance of each word in the context of the task, and selectively focus on the relevant parts of the context. This enhances the model’s ability to read and understand the context, ultimately improving its performance in named entity recognition tasks. Extensive experiments on three datasets, ACE2004, ACE2005, and GENIA, show that this method achieves superior performance when compared to previous methods.