English Named Entity Recognition Research Articles

Chinese Named Entity Recognition Based on Boundary Enhancement with Multi-Class Information

Compared with English named entity recognition (NER), Chinese NER faces significant challenges due to the flexible, non-standard word formation and vague word boundaries, which cause a lot of boundary ambiguity and reduce the accuracy of entity identification. To address this issue, we propose a boundary enhancement with multi-class information model (BEMCI). The model integrates multiple types of information into text embedding while enhancing the subsequent syntax-structure information. A syntactic information analysis module is designed to highlight important syntax information from three aspects, namely part-of-speech tags, syntactic constituents, and dependency relations, to analyze sentence structures. Meanwhile, an improved contextual attention mechanism, which combines contextual and syntactic information using a gate mechanism to control the weight fusion, is proposed to further enhance the model’s boundary determination. Multiple sets of experiments conducted on six general datasets show that BEMCI outperforms other baselines, achieving the best results in four of these six datasets.

Prompt-Based Word-Level Information Injection BERT for Chinese Named Entity Recognition

Named entity recognition (NER) is a subfield of natural language processing (NLP) that identifies and classifies entities from plain text, such as people, organizations, locations, and other types. NER is a fundamental task in information extraction, information retrieval, and text summarization, as it helps to organize the relevant information in a structured way. The current approaches to Chinese named entity recognition do not consider the category information of matched Chinese words, which limits their ability to capture the correlation between words. This makes Chinese NER more challenging than English NER, which already has well-defined word boundaries. To improve Chinese NER, it is necessary to develop new approaches that take into account category features of matched Chinese words, and the category information would help to effectively capture the relationship between words. This paper proposes a Prompt-based Word-level Information Injection BERT (PWII-BERT) to integrate prompt-guided lexicon information into a pre-trained language model. Specifically, we engineer a Word-level Information Injection Adapter (WIIA) through the original Transformer encoder and prompt-guided Transformer layers. Thus, the ability of PWII-BERT to explicitly obtain fine-grained character-to-word relevant information according to the category prompt is one of its key advantages. In experiments on four benchmark datasets, PWII-BERT outperforms the baselines, demonstrating the significance of fully utilizing the advantages of fusing the category information and lexicon feature to implement Chinese NER.

Entity Recognition for Chinese Hazardous Chemical Accident Data Based on Rules and a Pre-Trained Model

Due to the fragile physicochemical properties of hazardous chemicals, the chances of leakage and explosion during production, transportation, and storage are quite high. In recent years, hazardous chemical accidents have occurred frequently, posing a great threat to people’s lives and property. Hence, it is crucial to analyze hazardous chemical accidents and establish corresponding warning mechanisms and safeguard measures. At present, most hazardous-chemical-accident data exist in text format. However, named entity recognition (NER), as a method to extract useful information from text data, has not been fully utilized in the field of Chinese hazardous-chemical handling. The challenge is that Chinese NER is more difficult than English NER, because the boundaries of Chinese are fuzzy. In addition, the descriptions of hazardous chemical accidents are colloquial and lacks relevant labeling data. Further, most current models do not consider identifying the entities related to accident scenarios, losses, and causes. To tackle these issues, we propose a model based on a rule template and Bert-BiLSTM-CRF (RT-BBC) to recognize named entities from unstructured Chinese hazardous chemical accident reports. Comprehensive experiments on real-world datasets show the effectiveness of the proposed method. Specifically, RT-BBC outperformed the most competitive method by 6.6% and 3.6% in terms of accuracy and F1.

Improving sequence labeling with labeled clue sentences

Pre-trained language models (PLMs) have achieved noticeable success on a variety of natural language processing tasks, such as sequence labeling. In particular, the existing sequence labeling methods fine-tune PLMs on large-scale labeled data, which can avoid training the sequence labeling models from scratch. The fine-tuning process still requires large amounts of labeled training data so as to be effective. However, obtaining rich annotated data for sequence labeling is a time-consuming and expensive process, creating a substantial barrier for directly applying the PLMs trained on general-purpose large-scale text data to sequence labeling. In this paper, we investigate sequence labeling tasks from a novel perspective and propose a general framework that uses labeled clue sentences to mitigate the problem of insufficient annotation data for sequence labeling. Specifically, we first retrieve the labeled clue sentences for each original sentence in the training set based on the semantic (or syntactic) relevance. Here, the number of annotated clue sentences determines the expansion degree of the training set. Then, we modify the transformer’s self-attention mechanism to not only exploit the contextual information of the original sentence but also leverage the contextual and label information of the labeled clue sentences. In addition, we devise a mask label strategy to further avoid over-fitting by randomly masking out the labels of certain tokens in the clue sentence and then predicting these mask labels based on the context of the tokens corresponding to the mask labels. We verify the effectiveness and generalizability of the proposed framework on three sequence labeling tasks, including Chinese Named Entity Recognition, English Named Entity Recognition, and Aspect Term Extraction. Extensive experimental results show that our method can yield state-of-the-art or competitive results on the three tasks.

CL-NERIL: A Cross-Lingual Model for NER in Indian Languages (Student Abstract)

Developing Named Entity Recognition (NER) systems for Indian languages has been a long-standing challenge, mainly owing to the requirement of a large amount of annotated clean training instances. This paper proposes an end-to-end framework for NER for Indian languages in a low-resource setting by exploiting parallel corpora of English and Indian languages and an English NER dataset. The proposed framework includes an annotation projection method that combines word alignment score and NER tag prediction confidence score on source language (English) data to generate weakly labeled data in a target Indian language. We employ a variant of the Teacher-Student model and optimize it jointly on the pseudo labels of the Teacher model and predictions on the generated weakly labeled data. We also present manually annotated test sets for three Indian languages: Hindi, Bengali, and Gujarati. We evaluate the performance of the proposed framework on the test sets of the three Indian languages. Empirical results show a minimum 10% performance improvement compared to the zero-shot transfer learning model on all languages. This indicates that weakly labeled data generated using the proposed annotation projection method in target Indian languages can complement well-annotated source language data to enhance performance. Our code is publicly available at https://github.com/aksh555/CL-NERIL.

Leveraging Multi-source knowledge for Chinese clinical named entity recognition via relational graph convolutional network.

External knowledge, such as lexicon of words in Chinese and domain knowledge graph (KG) of concepts, has been recently adopted to improve the performance of machine learning methods for named entity recognition (NER) as it can provide additional information beyond context. However, most existing studies only consider knowledge from one source (i.e., either lexicon or knowledge graph) in different ways and consider lexicon words or KG concepts independently with their boundaries. In this paper, we focus on leveraging multi-source knowledge in a unified manner where lexicon words or KG concepts are well combined with their boundaries for Chinese Clinical NER (CNER). We propose a novel method based on relational graph convolutional network (RGCN), called MKRGCN, to utilize multi-source knowledge in a unified manner for CNER. For any sentence, a relational graph based on words or concepts in each knowledge source is constructed, where lexicon words or KG concepts appearing in the sentence are linked to the containing tokens with the boundary information of the lexicon words or KG concepts. RGCN is used to model all relational graphs constructed from multi-source knowledge, and the representations of tokens from multi-source knowledge are integrated into the context representations of tokens via an attention mechanism. Based on the knowledge-enhanced representations of tokens, we deploy a conditional random field (CRF) layer for named entity label prediction. In this study, a lexicon of words and a medical knowledge graph are used as knowledge sources for Chinese CNER. Our proposed method achieves the best performance on CCKS2017 and CCKS2018 in Chinese with F1-scores of 91.88% and 89.91%, respectively, significantly outperforming existing methods. The extended experiments on NCBI-Disease and BC2GM in English also prove the effectiveness of our method when only considering one knowledge source via RGCN. The MKRGCN model can integrate knowledge from the external lexicon and knowledge graph effectively for Chinese CNER and has the potential to be applied to English NER.

A Residual BiLSTM Model for Named Entity Recognition

As one of the most powerful neural networks, Long Short-Term Memory (LSTM) is widely used in natural language processing (NLP) tasks. Meanwhile, the BiLSTM-CRF model is one of the most popular models for named entity recognition (NER), and many state-of-the-art models for NER are based on it. In this paper, we propose a new residual BiLSTM model and perform it with a conditional random field (CRF) layer together on NER tasks. Based on the most popular BiLSTM-CRF model, we replace the BiLSTM with our residual BiLSTM blocks to encode words or characters. We evaluate our model on Chinese and English datasets. We utilize both word2vec and BERT to generate word or character vectors. Furthermore, we conduct experiments to compare the performance of NER by using different structures of residual blocks. The experimental results show that our model can improve the performance of both Chinese and English NER effectively without introducing any external knowledge.

Learning to select pseudo labels: a semi-supervised method for named entity recognition

Deep learning models have achieved state-of-the-art performance in named entity recognition (NER); the good performance, however, relies heavily on substantial amounts of labeled data. In some specific areas such as medical, financial, and military domains, labeled data is very scarce, while unlabeled data is readily available. Previous studies have used unlabeled data to enrich word representations, but a large amount of entity information in unlabeled data is neglected, which may be beneficial to the NER task. In this study, we propose a semi-supervised method for NER tasks, which learns to create high-quality labeled data by applying a pre-trained module to filter out erroneous pseudo labels. Pseudo labels are automatically generated for unlabeled data and used as if they were true labels. Our semi-supervised framework includes three steps: constructing an optimal single neural model for a specific NER task, learning a module that evaluates pseudo labels, and creating new labeled data and improving the NER model iteratively. Experimental results on two English NER tasks and one Chinese clinical NER task demonstrate that our method further improves the performance of the best single neural model. Even when we use only pre-trained static word embeddings and do not rely on any external knowledge, our method achieves comparable performance to those state-of-the-art models on the CoNLL-2003 and OntoNotes 5.0 English NER tasks.

A parallel computing-based Deep Attention model for named entity recognition

Named entity recognition (NER) is an important task in natural language processing and has been widely studied. In recent years, end-to-end NER with bidirectional long short-term memory (BiLSTM) has received more and more attention. However, it remains a major challenge for BiLSTM to parallel computing, long-range dependencies and single feature space mapping. We propose a deep neural network model which is based on parallel computing self-attention mechanism to address these problems. We only use a small number of BiLSTMs to capture the time series of texts and then make use of self-attention mechanism that allows parallel computing to capture long-range dependencies. Experiments on two NER datasets show that our model is superior in quality and takes less training time. Our model achieves an F1 score of 92.63% on the SIGHAN bakeoff 2006 MSRA portion for Chinese NER, improving over the existing best results by over 1.4%. On the CoNLL2003 shared task portion for English NER, our model achieves an F1 score of 92.17%, which outperforms the previous state-of-the-art results by 0.91%.

Malay Named Entity Recognition Using Rule Based Approach

Named Entity Recognition (NER) research based on rule is widely investigated and is used in various languages mainly English. However, the English NER rules are different with Malay language due to different morphology. Some of challenging issue in Malay is cross reference between named entities, and entity repetition. This paper proposes to solve the issues in Malay NER. This study starts by providing Malay online news corpus, gazeteer development, rules development and evaluation. This study focus on nine name entities i.e person, organization, position, date, time, currency, measurement and percentage. Overall the experimental result shows 90.23% precision, 92.13% recall and 91.05% f-measure. The outcome from this research is expected to help other researchers in implementing the Malay NER using rule based approach through the addition of new rules to achieve higher accuracy.

A neural-based re-ranking model for Chinese named entity recognition

Chinese named entity recognition (CNER) is different from English named entity recognition (ENER). There is no specific delimiter in Chinese text to determine the words in a sentence. Besides, the combination of Chinese text has a strong arbitrariness. These special cases usually bring more errors to the Chinese NER (CNER). We propose a re-ranking model based on BILSTM network and without using any other auxiliary methods. Our approach uses N-best generalised label sequences that are produced by baseline model as input and feeds them into our re-ranking model for modelling the context within the generalised sequences. The optimal output sequence is obtained by comprehensively considering the result of baseline model and re-ranking model. Experimental results show that our model achieves better F1-score on Bakeoff-3 MSRA corpus than the best previous experimental results, which yields a 0.97% improvement on F1-score over our neural baseline model and a 0.22% improvement over the state-of-the-art CNER model.

A neural-based re-ranking model for Chinese named entity recognition

Chinese named entity recognition (CNER) is different from English named entity recognition (ENER). There is no specific delimiter in Chinese text to determine the words in a sentence. Besides, the combination of Chinese text has a strong arbitrariness. These special cases usually bring more errors to the Chinese NER (CNER). We propose a re-ranking model based on BILSTM network and without using any other auxiliary methods. Our approach uses N-best generalised label sequences that are produced by baseline model as input and feeds them into our re-ranking model for modelling the context within the generalised sequences. The optimal output sequence is obtained by comprehensively considering the result of baseline model and re-ranking model. Experimental results show that our model achieves better F1-score on Bakeoff-3 MSRA corpus than the best previous experimental results, which yields a 0.97% improvement on F1-score over our neural baseline model and a 0.22% improvement over the state-of-the-art CNER model.

Malay Named Entity Recognition Based on Rule-Based Approach

A Named-Entity Recognition (NER) is part of the process in Text Mining and it is a very useful process for information extraction. This NER tool can be used to assist user in identifying and detecting entities such as person, location or organization. However, different languages may have different morphologies and thus require different NER processes. For instance, an English NER process cannot be applied in processing Malay articles due to the different morphology used in different languages. This paper proposes a Rule-Based Named-Entity Recognition algorithm for Malay articles. The proposed Malay NER is designed based on a Malay part-of-speech (POS) tagging features and contextual features that had been implemented to handle Malay articles. Based on the POS results, proper names will be identified or detected as the possible candidates for annotation. Besides that, there are some symbols and conjunctions that will also be considered in the process of identifying named-entity for Malay articles. Several manually constructed dictionaries will be used to handle three named-entities; Person, Location and Organizations. The experimental results show a reasonable output of 89.47% for the F-Measure value. The proposed Malay NER algorithm can be further improved by having more complete dictionaries and refined rules to be used in order to identify the correct Malay entities system.