Chinese Named Entity Recognition Task Research Articles

Named Entity Recognition for Public Interest Litigation Based on a Deep Contextualized Pretraining Approach

The named entity recognition (NER) in the field of public interest litigation can assist prosecutors in handling cases and provide them with specific entities in making legal documents. Previously, the context-free deep learning model is used to catch the semantic comprehension, in which the static word vector is obtained without considering the context. Moreover, this kind of method relies on word segmentation technology and cannot solve the error transmission caused by word segmentation inaccuracy, which brings great challenges to the Chinese NER task. To tackle the above issues, an entity recognition method based on pretraining is proposed. First, based on the basic entities, three legal ontologies, NERP, NERCGP, and NERFPP are developed to expand the named entity recognition corpus in the judicial field. Second, a variant of the pretrained model BERT (Bidirectional Encoder Representations from Transformer) called BERT-WWM (whole-word mask)-EXT(extra) is introduced to catch the text character-level word vector hierarchical and the context bidirectional features, which effectively solve the problem of task boundary division of named entities. Then, to further improve the model recognition effect, the general knowledge learned from the pretrained model is used to fit the downstream neural network BiLSTM (bi-long short-term memory), and at the end of the architecture, CRF (conditional random fields) is introduced to restrict the label relationship. Finally, the experimental results show that the proposed method is more effective than the existing methods, which reach 96% and 90% in the F1 index of NER and NERP entities, respectively.

Pronounce differently, mean differently: A multi-tagging-scheme learning method for Chinese NER integrated with lexicon and phonetic features

Named Entity Recognition (NER) aims to automatically extract specific entities from the unstructured text. Compared with performing NER in English, Chinese NER is more challenging in recognizing entity boundaries because there are no explicit delimiters between Chinese characters. However, most previous researches focused on the semantic information of the Chinese language on the character level but ignored the importance of the phonetic characteristics. To address these issues, we integrated phonetic features of Chinese characters with the lexicon information to help disambiguate the entity boundary recognition by fully exploring the potential of Chinese as a pictophonetic language. In addition, a novel multi-tagging-scheme learning method was proposed, based on the multi-task learning paradigm, to alleviate the data sparsity and error propagation problems that occurred in the previous tagging schemes, by separately annotating the segmentation information of entities and their corresponding entity types. Extensive experiments performed on four Chinese NER benchmark datasets: OntoNotes4.0, MSRA, Resume, and Weibo, show that our proposed method consistently outperforms the existing state-of-the-art baseline models. The ablation experiments further demonstrated that the introduction of the phonetic feature and the multi-tagging-scheme has a significant positive effect on the improvement of the Chinese NER task.

Leveraging Integrated Learning for Open-Domain Chinese Named Entity Recognition

Named entity recognition (NER) is a fundamental technique in natural language processing that provides preconditions for tasks, such as natural language question reasoning, text matching, and semantic text similarity. Compared to English, the challenge of Chinese NER lies in the noise impact caused by the complex meanings, diverse structures, and ambiguous semantic boundaries of the Chinese language itself. At the same time, compared with specific domains, open-domain entity types are more complex and changeable, and the number of entities is considerably larger. Thus, the task of Chinese NER is more difficult. However, existing open-domain NER methods have low recognition rates. Therefore, this paper proposes a method based on the bidirectional long short-term memory conditional random field (BiLSTM-CRF) model, which leverages integrated learning to improve the efficiency of Chinese NER. Compared with single models, including CRF, BiLSTM-CRF, and gated recurrent unit-CRF, the proposed method can significantly improve the accuracy of open-domain Chinese NER.

Chinese Named Entity Recognition Based on Knowledge Based Question Answering System

The KBQA (Knowledge-Based Question Answering) system is an essential part of the smart customer service system. KBQA is a type of QA (Question Answering) system based on KB (Knowledge Base). It aims to automatically answer natural language questions by retrieving structured data stored in the knowledge base. Generally, when a KBQA system receives the user’s query, it first needs to recognize topic entities of the query, such as name, location, organization, etc. This process is the NER (Named Entity Recognition). In this paper, we use the Bidirectional Long Short-Term Memory-Conditional Random Field (Bi-LSTM-CRF) model and introduce the SoftLexicon method for a Chinese NER task. At the same time, according to the analysis of the characteristics of application scenario, we propose a fuzzy matching module based on the combination of multiple methods. This module can efficiently modify the error recognition results, which can further improve the performance of entity recognition. We combine the NER model and the fuzzy matching module into an NER system. To explore the availability of the system in some specific fields, such as a power grid field, we utilize the power grid-related original data collected by the Hebei Electric Power Company to improve our system according to the characteristics of data in the power grid field. We innovatively make the dataset and high-frequency word lexicon in the power grid field, which makes our proposed NER system perform better in recognizing entities in the field of power grid. We used the cross-validation method for validation. The experimental results show that the F1-score of the improved NER model on the power grid dataset reaches 92.43%. After processing the recognition results by using the fuzzy matching module, about 99% of the entities in the test set can be correctly recognized. It proves that the proposed NER system can achieve excellent performance in the application scenario of a power grid. The results of this work will also fill the gap in the research of intelligent customer-service-related technologies in the power grid field in China.

Dependency syntax guided BERT-BiLSTM-GAM-CRF for Chinese NER

Traditional named entity recognition (NER) methods derived from deep learning (DL) often ignore the long-distance syntactic dependence relationships between words. In such methods, the representation of Chinese word vectors is too simple to solve the problem of polysemy. To address these shortcomings, this paper proposes a syntactic dependency guided BERT-BiLSTM-GAM-CRF model for Chinese NER. First, the self-attention method guided by the dependency syntactic parsing tree is embedded in the transformer computing framework of the BERT model. This can not only obtain the deep two-way linguistic representation of a word according to the context information of the word, but it can also better express the long-distance syntactic dependency relationships between words; Second, the trained word vector sequence is input into the BiLSTM layer embedded in the global attention mechanism (GAM), and then the most important whole situation semantic information in the sentence is obtained. Finally, the CRF is employed to learn the dependence relationships between adjacent labels to obtain the best sentence level label sequence. A large number of experiments on the CLUENER-2020 corpus, MSRA corpus, Weibo corpus and OntoNotes4 corpus prove that the constructed model has good results for the Chinese NER task, and the F1 values are 81.08%, 94.97%, 63.60% and 75.64% respectively.

A review of Chinese named entity recognition

Named Entity Recognition (NER) is used to identify entity nouns in the corpus such as Location, Person and Organization, etc. NER is also an important basic of research in various natural language fields. The processing of Chinese NER has some unique difficulties, for example, there is no obvious segmentation boundary between each Chinese character in a Chinese sentence. The Chinese NER task is often combined with Chinese word segmentation, and so on. In response to these problems, we summarize the recognition methods of Chinese NER. In this review, we first introduce the sequence labeling system and evaluation metrics of NER. Then, we divide Chinese NER methods into rule-based methods, statistics-based machine learning methods and deep learning-based methods. Subsequently, we analyze in detail the model framework based on deep learning and the typical Chinese NER methods. Finally, we put forward the current challenges and future research directions of Chinese NER technology.

CWPC_BiAtt: Character–Word–Position Combined BiLSTM-Attention for Chinese Named Entity Recognition

Usually taken as linguistic features by Part-Of-Speech (POS) tagging, Named Entity Recognition (NER) is a major task in Natural Language Processing (NLP). In this paper, we put forward a new comprehensive-embedding, considering three aspects, namely character-embedding, word-embedding, and pos-embedding stitched in the order we give, and thus get their dependencies, based on which we propose a new Character–Word–Position Combined BiLSTM-Attention (CWPC_BiAtt) for the Chinese NER task. Comprehensive-embedding via the Bidirectional Llong Short-Term Memory (BiLSTM) layer can get the connection between the historical and future information, and then employ the attention mechanism to capture the connection between the content of the sentence at the current position and that at any location. Finally, we utilize Conditional Random Field (CRF) to decode the entire tagging sequence. Experiments show that CWPC_BiAtt model we proposed is well qualified for the NER task on Microsoft Research Asia (MSRA) dataset and Weibo NER corpus. A high precision and recall were obtained, which verified the stability of the model. Position-embedding in comprehensive-embedding can compensate for attention-mechanism to provide position information for the disordered sequence, which shows that comprehensive-embedding has completeness. Looking at the entire model, our proposed CWPC_BiAtt has three distinct characteristics: completeness, simplicity, and stability. Our proposed CWPC_BiAtt model achieved the highest F-score, achieving the state-of-the-art performance in the MSRA dataset and Weibo NER corpus.

BiLSTM-CRF Chinese Named Entity Recognition Model with Attention Mechanism

In order to make up for the weakness of insufficient considering dependency of the input char sequence in the deep learning method of Chinese named entity recognition task, this paper proposes a method, which integrate Bidirectional Long Short-Term Memory (BiLSTM), attention mechanism and add the information of word vector. Firstly, the proposed model obtains the char vector feature extracted from the text corpus, which is then input to the BiLSTM model; Secondly, the attention mechanism is used to calculate the relevance between the current input char and the other input char of the BiLSTM model; Finally, the global feature is obtained according to relevance, concatenating the word vector feature, which is introduced to the Conditional random field(CRF) layer to perform the mutual constraint between tags. Thus, the classified result can be obtained. Based on the corpus of the Chinese Peoples’ Daily Newspaper in 1998, our experiments show that the proposed method can improved the performance and efficiency of named entity recognition, compared to the existing deep-learning method that combines word vector and char vector.

TL-NER: A Transfer Learning Model for Chinese Named Entity Recognition

Most of the current research on Named Entity Recognition (NER) in the Chinese domain is based on the assumption that annotated data are adequate. However, in many scenarios, the sufficient amount of annotated data required for Chinese NER task is difficult to obtain, resulting in poor performance of machine learning methods. In view of this situation, this paper tries to excavate the information contained in the massive unlabeled raw text data and utilize it to enhance the performance of Chinese NER task. A deep learning model combined with Transfer Learning technique is proposed in this paper. This method can be leveraged in some domains where there is a large amount of unlabeled text data and a small amount of annotated data. The experiment results show that the proposed method performs well on different sized datasets, and this method also avoids errors that occur during the word segmentation process. We also evaluate the effect of transfer learning from different aspects through a series of experiments.

LSTM-CRF Neural Network With Gated Self Attention for Chinese NER

Named entity recognition (NER) is an essential part of natural language processing tasks. Chinese NER task is different from the many European languages due to the lack of natural delimiters. Therefore, Chinese Word Segmentation (CWS) is usually regarded as the first step of processing Chinese NER. However, the word-based NER models relying on CWS are more vulnerable to incorrectly segmented entity boundaries and the presence of out-of-vocabulary (OOV) words. In this paper, we propose a novel character-based Gated Convolutional Recurrent neural network with Attention called GCRA for Chinese NER task. In particular, we introduce a hybrid convolutional neural network with gating filter mechanism to capture local context information and a highway neural network after LSTM to select characters of interest. The additional gated self-attention mechanism is used to capture the global dependencies from different multiple subspaces and arbitrary adjacent characters. We evaluate the performance of our proposed model on three datasets, including SIGHAN bakeoff 2006 MSRA, Chinese Resume, and Literature NER dataset. The experiment results show that our model outperforms other state-of-the-art models without relying on any external resources like lexicons and multi-task joint training.

Chinese NER Using Dynamic Meta-Embeddings

Named entity recognition (NER) is one of the basic techniques in natural language processing tasks. Chinese NER is complicated and difficult which remains a major challenge. One of the main reasons is that the boundaries of entities in Chinese are blurred and closely related to word segmentation results. Previous studies for this task have broadly divided into two categories, word-based, and character-based methods. However, the former class suffers from the word segmentation errors, and the latter cannot make full use of the information on multiple granularities. To address these problems, we investigate a new dynamic meta-embeddings method and apply it to Chinese NER task, which utilizes attention mechanism to combine features of both character and word granularity in embedding layer. The meta-embeddings created by our method are dynamic, data-specific, and task-specific, since the meta-embeddings for same characters in different sentence sequences are distinct. The experiments on MSRA and LiteratureNER datasets validate the effectiveness of our model, and this method achieves the state-of-the-art results on LiteratureNER.

Improving Named Entity Recognition using Bilingual Constraints and Word Alignment

Named entities carry essential meanings and information in natural language. Therefore, Named Entity Recognition has many applications in different Natural Language Processing tasks such as Information Retrieval, Information Extraction, Machine Translation, and Question Answering. State-of-the-art Named Entity Recognition systems are based on supervised machine learning algorithms which require huge amounts of training data. The main problem, however, is that constructing named entity annotated corpora is an expensive, labor-intensive, and time-consuming task. Therefore, in this paper, we propose an approach to improve monolingual Named Entity Recognition systems by exploiting an existing unannotated English-Chinese bilingual corpus. The system jointly recognizes named entities in both English and Chinese sentences through the use of bilingual constraints. Experimental results show an improvement in Named Entity Recognition of both Chinese and English compared to the strong baseline StanfordNER. In particular, Chinese Named Entity Recognition improves significantly by 20.81% in term of F1-score. As for the English language, Named Entity Recognition F1-score increases slightly from 75.75% to 76.08%. When comparing to the state-of-the-art system in improving Named Entity Recognition based on bilingual resources, we manage to outperform in Chinese Named Entity Recognition task by 5.99% and achieve comparable results for the English side. Our proposed method can also be generalized to apply to resource-limited languages.

Boosted Web Named Entity Recognition via Tri-Training

Named entity extraction is a fundamental task for many natural language processing applications on the web. Existing studies rely on annotated training data, which is quite expensive to obtain large datasets, limiting the effectiveness of recognition. In this research, we propose a semisupervised learning approach for web named entity recognition (NER) model construction via automatic labeling and tri-training. The former utilizes structured resources containing known named entities for automatic labeling, while the latter makes use of unlabeled examples to improve the extraction performance. Since this automatically labeled training data may contain noise, a self-testing procedure is used as a follow-up to remove low-confidence annotation and prepare higher-quality training data. Furthermore, we modify tri-training for sequence labeling and derive a proper initialization for large dataset training to improve entity recognition. Finally, we apply this semisupervised learning framework for person name recognition, business organization name recognition, and location name extraction. In the task of Chinese NER, an F-measure of 0.911, 0.849, and 0.845 can be achieved, for person, business organization, and location NER, respectively. The same framework is also applied for English and Japanese business organization name recognition and obtains models with performance of a 0.832 and 0.803 F-measure.