Nested Named Entity Recognition Research Articles

Nested Entity Recognition Method Based on Multidimensional Features and Fuzzy Localization

Nested named entity recognition (NNER) aims to identify potentially overlapping named entities. Sequence labeling method and span-based method are two commonly used methods in nested named entity recognition. However, the linear structure of sequence labeling method results in relatively poor performance, and span-based method requires traversing all spans, which brings very high time complexity. All of them fail to effectively leverage the positional dependencies between internal and external entities. In order to improve these issues, this paper proposed a nested entity recognition method based on Multidimensional Features and Fuzzy Localization (MFFL). Firstly, this method adopted the shared encoding that fused three features of characters, words, and parts of speech to obtain a multidimensional feature vector representation of the text and obtained rich semantic information in the text. Secondly, we proposed to use the fuzzy localization to assist the model in pinpointing the potential locations of entities. Finally, in the entity classification, it used a window to expand the sub-sequence and enumerate possible candidate entities and predicted the classification labels of these candidate entities. In order to alleviate the problem of error propagation and effectively learn the correlation between fuzzy localization and classification labels, we adopted multi-task learning strategy. This paper conducted several experiments on two public datasets. The experimental results showed that the proposed method achieves ideal results in both nested entity recognition and non-nested entity recognition tasks, and significantly reduced the time complexity of nested entity recognition.

Recursive label attention network for nested named entity recognition

Nested named entity recognition (NER) has been mainly studied by recognizing inner entities first, then outer entities, or vice versa, depending on the nesting level. However, the previous models on nested NER do not deal with the data sparseness problems of deep nested entities (i.e., insufficiency of training data for deeper nested entities) and the error propagation problems between nested entities (i.e., accumulation of errors caused by inner entities or outer entities). To alleviate the data sparseness problems, we propose a recursive label attention network. In contrast to previous models, the proposed model explicitly reflects the nested level and effectively re-uses lower level label information through level-reflected label embeddings. To counteract error propagation in the recursive architecture, we propose an inner entity pretraining strategy in which the proposed model is sequentially trained from lower to higher levels. In experiments, we use task-specific metrics for nested NER, unlike the evaluation metrics used in previous studies. The experimental results show that the proposed model is particularly effective in cases where entities of the same type were nested. In addition, it showed high F1-scores even when named entities are deeply nested. This fact reveals that our proposed model is effective for nest NER.

Research on Chinese Nested Entity Recognition Based on IDCNNLR and GlobalPointer

The task of named entity recognition (NER) is to identify entities in the text and predict their categories. In real-life scenarios, the context of the text is often complex, and there may exist nested entities within an entity. This kind of entity is called a nested entity, and the task of recognizing entities with nested structures is referred to as nested named entity recognition. Most existing NER models can only handle flat entities, and there has been limited research progress in Chinese nested named entity recognition, resulting in relatively few models in this direction. General NER models have limited semantic extraction capabilities and cannot capture deep semantic information between nested entities in the text. To address these issues, this paper proposes a model that uses the GlobalPointer module to identify nested entities in the text and constructs the IDCNNLR semantic extraction module to extract deep semantic information. Furthermore, multiple-head self-attention mechanisms are incorporated into the model at multiple positions to achieve data denoising, enhancing the quality of semantic features. The proposed model considers each possible entity boundary through the GlobalPointer module, and the IDCNNLR semantic extraction module and multi-position attention mechanism are introduced to enhance the model’s semantic extraction capability. Experimental results demonstrate that the proposed model achieves F1 scores of 69.617% and 79.285% on the CMeEE Chinese nested entity recognition dataset and CLUENER2020 Chinese fine-grained entity recognition dataset, respectively. The model exhibits improvement compared to baseline models, and each innovation point shows effective performance enhancement in ablative experiments.

SinaTools: Open Source Toolkit for Arabic Natural Language Processing

We introduce SinaTools, an open-source Python package for Arabic natural language processing and understanding. SinaTools is a unified package allowing people to integrate it into their system workflow, offering solutions for various tasks such as flat and nested Named Entity Recognition (NER), fully-flagged Word Sense Disambiguation (WSD), Semantic Relatedness, Synonymy Extractions and Evaluation, Lemmatization, Part-of-speech Tagging, Root Tagging, and additional helper utilities such as corpus processing, text stripping methods, and diacritic-aware word matching. This paper presents SinaTools and its benchmarking results, demonstrating that SinaTools outperforms all similar tools on the aforementioned tasks, such as Flat NER (87.33%), Nested NER (89.42%), WSD (82.63%), Semantic Relatedness (0.49 Spearman rank), Lemmatization (90.5%), POS tagging (93.8%), among others. SinaTools can be downloaded from (https://sina.birzeit.edu/sinatools).

A Chinese nested named entity recognition approach using sequence labeling

PurposeThis study aims to introduce an innovative approach that uses a decoder with multiple layers to accurately identify Chinese nested entities across various nesting depths. To address potential human intervention, an advanced optimization algorithm is used to fine-tune the decoder based on the depth of nested entities present in the data set. With this approach, this study achieves remarkable performance in recognizing Chinese nested entities.Design/methodology/approachThis study provides a framework for Chinese nested named entity recognition (NER) based on sequence labeling methods. Similar to existing approaches, the framework uses an advanced pre-training model as the backbone to extract semantic features from the text. Then a decoder comprising multiple conditional random field (CRF) algorithms is used to learn the associations between granularity labels. To minimize the need for manual intervention, the Jaya algorithm is used to optimize the number of CRF layers. Experimental results validate the effectiveness of the proposed approach, demonstrating its superior performance on both Chinese nested NER and flat NER tasks.FindingsThe experimental findings illustrate that the proposed methodology can achieve a remarkable 4.32% advancement in nested NER performance on the People’s Daily corpus compared to existing models.Originality/valueThis study explores a Chinese NER methodology based on the sequence labeling ideology for recognizing sophisticated Chinese nested entities with remarkable accuracy.

Handling negative samples problems in span-based nested named entity recognition

Named Entity Recognition (NER) is generally regarded as a sequence labeling task, and faces a serious problem when the named entities are nested. Span-based model, which enumerates all possible spans as potential entity mentions in a sentence and classifies them, is straightforward for nested NER but faces negative samples problems. In this paper, we propose a span-based nested NER model with BERT and try to solve the negative samples problems. In view of the phenomenon that there are too many negative samples in all spans, we employ a multi-task learning method, which divides NER task into entity identification and entity classification task. In addition, we propose the entity IoU loss function to focus our model on the hard negative samples. Our model is evaluated on three nested NER datasets: GENIA, ACE2004 and ACE2005, and the results show that our model outperforms other state-of-the-art models with the same pretrained language model, achieving 79.46%, 87.30% and 85.24% respectively in terms of F1 score.

Candidate region aware nested named entity recognition

Named entity recognition (NER) is crucial in various natural language processing (NLP) tasks. However, the nested entities which are common in practical corpus are often ignored in most of current NER models. To extract the nested entities, two categories of models (i.e., feature-based and neural network-based approaches) are proposed. However, the feature-based models suffer from the complicated feature engineering and often heavily rely on the external resources. Discarding the heavy feature engineering, recent neural network-based methods which treat the nested NER as a classification task are designed but still suffer from the heavy class imbalance issue and the high computational cost. To solve these problems, we propose a neural multi-task model with two modules: Binary Sequence Labeling and Candidate Region Classification to extract the nested entities. Extensive experiments are conducted on the public datasets. Comparing with recent neural network-based approaches, our proposed model achieves the better performance and obtains the higher efficiency.

Nested Named Entity Recognition via an Independent-Layered Pretrained Model

When an entity contains one or more entities, these particular entities are referred to as nested entities. The Layered BiLSTM-CRF model can use multiple BiLSTM layers to identify nested entities. However, as the number of layers increases, the number of labels that the model can learn decreases, and it may not even predict any entities, thereby causing the model to stop stacking. Furthermore, the model will be constrained by the one-way propagation of information from the lower layer to the higher layer. The incorrect entities extracted by the outer layer will affect the performance of the inner layer. We propose a novel neural network for nested named entity recognition (NER) that dynamically stacks flat NER layers to address these issues. Each flat NER layer captures contextual information based on a pretrained model with more robust feature extraction capabilities. The model parameters of a flat NER layer and its input are entirely independent. The input of each layer is all of the word representations generated by the input sequence through the embedding layer. The independent input ensures that different flat NER layers will not be interfered with by other flat NER layers during model training and testing to reduce error propagation. Experiments show that our model obtains F1 scores of 76.9%, 78.1%, and 78.0% on the ACE2004, ACE2005, and GENIA datasets, respectively.

Syntactically-informed word representations from graph neural network

Most deep language understanding models depend only on word representations, which are mainly based on language modelling derived from a large amount of raw text. These models encode distributional knowledge without considering syntactic structural information, although several studies have shown benefits of including such information. Therefore, we propose new syntactically-informed word representations (SIWRs), which allow us to enrich the pre-trained word representations with syntactic information without training language models from scratch. To obtain SIWRs, a graph-based neural model is built on top of either static or contextualised word representations such as GloVe, ELMo and BERT. The model is first pre-trained with only a relatively modest amount of task-independent data that are automatically annotated using existing syntactic tools. SIWRs are then obtained by applying the model to downstream task data and extracting the intermediate word representations. We finally replace word representations in downstream models with SIWRs for applications. We evaluate SIWRs on three information extraction tasks, namely nested named entity recognition (NER), binary and n-ary relation extractions (REs). The results demonstrate that our SIWRs yield performance gains over the base representations in these NLP tasks with 3–9% relative error reduction. Our SIWRs also perform better than fine-tuning BERT in binary RE. We also conduct extensive experiments to analyse the proposed method.

A Bidirectional Iterative Algorithm for Nested Named Entity Recognition

Nested named entity recognition (NER) is a special case of structured prediction in which annotated sequences can be contained inside each other. It is a challenging and significant problem in natural language processing. In this paper, we propose a novel framework for nested named entity recognition tasks. Our approach is based on a deep learning model which can be called in an iterative way, expanding the set of predicted entity mentions with each subsequent iteration. The proposed framework combines two such models trained to identify named entities in different directions: from general to specific ( outside-in ), and from specific to general ( inside-out ). The predictions of both models are then aggregated by a selection policy. We propose and evaluate several selection policies which can be used with our algorithm. Our method does not impose any restrictions on the length of entity mentions, number of entity classes, depth, or structure of the predicted output. The framework has been validated experimentally on four well-known nested named entity recognition datasets: GENIA, NNE, PolEval, and GermEval. The datasets differ in terms of domain (biomedical, news, mixed), language (English, Polish, German), and the structure of nesting (simple, complex). Through extensive tests, we prove that the approach we have proposed outperforms existing methods for nested named entity recognition.

Joint Learning of Token Context and Span Feature for Span-Based Nested NER

Nested named entity recognition (NER) is a linguistic phenomenon that has received increasing attention in the NLP community. In this article, we propose a novel joint learning network of the token context, and span feature (TCSF) for nested NER. Our model is a combination of token context network (TCN) for token learning, and deep residual convolutional neural network (CNN), and span relation network (SRN) for span learning. Span features are represented at both the token level, and span relation level. The span relation representation of SRN is trained on the similarity of span, and its positional features by attention weights. The $IoU$ metric is employed for span filtering, and analysis of the overlapping adjacent spans. Moreover, we propose a novel head-inner-tail (HI $^{P}$ T) operation to extend the inner features in spans for a fixed-length representation. TCSF is a fully end-to-end entity recognition model. We perform a set of comprehensive experiments, including an ablation study on the ACE, and GENIA datasets, and show state-of-the-art performance without, and with the pretrained language models.