Fine-grained Named Entity Recognition Research Articles

Chinese Fine-grained Named Entity Recognition based on Deep Learning

Chinese Fine-grained Named Entity Recognition based on Deep Learning

Chinese Fine-Grained Named Entity Recognition Based on BILTAR and GlobalPointer Modules

The task of fine-grained named entity recognition is to locate entities in text and classify them into predefined fine-grained categories. At present, Chinese fine-grained NER only uses the pretrained language model to encode the characters in the sentence and lacks the ability to extract the deep semantic, sequence, and position information. The sequence annotation method is character-based and lacks the processing of entity boundaries. Fine-grained entity categories have a high degree of similarity, which makes it difficult to distinguish similar categories. To solve the above problems, this paper constructs the BILTAR deep semantic extraction module and adds the GlobalPointer module to improve the accuracy of Chinese fine-grained named entity recognition. The BILTAR module is used to extract deep semantic features from the coding information of pretrained language models and use higher-quality features to improve the model performance. In the GlobalPointer module, the model first adds the rotation position encoding information to the feature vector, using the position information to achieve data enhancement. Finally, the model considers all possible entity boundaries through the GlobalPointer module and calculates the scores for all possible entity boundaries in each category. In this paper, all possible entity boundaries in the text are considered by the above method, and the accuracy of entity recognition is improved. In this paper, the corresponding experiments were carried out on CLUENER 2020 and the micro Chinese fine-grained NER dataset, and the F1 scores of the model in this paper reached 80.848% and 75.751%, respectively. In ablation experiments, the proposed method outperforms the most advanced baseline model and improves the performance of the basic model.

Application of Deep Learning in Generating Structured Radiology Reports: A Transformer-Based Technique.

Since radiology reports needed for clinical practice and research are written and stored in free-text narrations, extraction of relative information for further analysis is difficult. In these circumstances, natural language processing (NLP) techniques can facilitate automatic information extraction and transformation of free-text formats to structured data. In recent years, deep learning (DL)-based models have been adapted for NLP experiments with promising results. Despite the significant potential of DL models based on artificial neural networks (ANN) and convolutional neural networks (CNN), the models face some limitations to implement in clinical practice. Transformers, another new DL architecture, have been increasingly applied to improve the process. Therefore, in this study, we propose a transformer-based fine-grained named entity recognition (NER) architecture for clinical information extraction. We collected 88 abdominopelvic sonography reports in free-text formats and annotated them based on our developed information schema. The text-to-text transfer transformer model (T5) and Scifive, a pre-trained domain-specific adaptation of the T5 model, were applied for fine-tuning to extract entities and relations and transform the input into a structured format. Our transformer-based model in this study outperformed previously applied approaches such as ANN and CNN models based on ROUGE-1, ROUGE-2, ROUGE-L, and BLEU scores of 0.816, 0.668, 0.528, and 0.743, respectively, while providing an interpretable structured report.

Noun-based attention mechanism for Fine-grained Named Entity Recognition

Fine-grained Named Entity Recognition is a challenging Natural Language Processing problem as it requires classifying entity mentions into hundreds of types that can span across several domains and be organized in several hierarchy levels. This task can be divided into two subtasks: Fine-grained Named Entity Detection and Fine-grained Named Entity Typing. In this work, we propose solutions for both of these subtasks. For the former, we propose a system that uses a stack of Byte-Pair Encoded vectors in combination with Flair embeddings, followed by a BILSTM-CRF network, which allowed us to improve the current state of the art for the 1k-WFB-g dataset. In the second subtask, attention mechanisms have become a common component in most of the current architectures, where the patterns captured by these mechanisms are generic, so in theory, they could attend to any word in the text indistinctly, regardless of its syntactic type, often causing inexplicable errors. To overcome this limitation we propose an attention mechanism based specifically on the use of elements of the noun syntactic type. We have compared our results to those obtained with a generic attention mechanism, where our method presented better results.

Fine-Grained Named Entity Recognition Using a Multi-Stacked Feature Fusion and Dual-Stacked Output in Korean

Named entity recognition (NER) is a natural language processing task to identify spans that mention named entities and to annotate them with predefined named entity classes. Although many NER models based on machine learning have been proposed, their performance in terms of processing fine-grained NER tasks was less than acceptable. This is because the training data of a fine-grained NER task is much more unbalanced than those of a coarse-grained NER task. To overcome the problem presented by unbalanced data, we propose a fine-grained NER model that compensates for the sparseness of fine-grained NEs by using the contextual information of coarse-grained NEs. From another viewpoint, many NER models have used different levels of features, such as part-of-speech tags and gazetteer look-up results, in a nonhierarchical manner. Unfortunately, these models experience the feature interference problem. Our solution to this problem is to adopt a multi-stacked feature fusion scheme, which accepts different levels of features as its input. The proposed model is based on multi-stacked long short-term memories (LSTMs) with a multi-stacked feature fusion layer for acquiring multilevel embeddings and a dual-stacked output layer for predicting fine-grained NEs based on the categorical information of coarse-grained NEs. Our experiments indicate that the proposed model is capable of state-of-the-art performance. The results show that the proposed model can effectively alleviate the unbalanced data problem that frequently occurs in a fine-grained NER task. In addition, the multi-stacked feature fusion layer contributes to the improvement of NER performance, confirming that the proposed model can alleviate the feature interference problem. Based on this experimental result, we conclude that the proposed model is well-designed to effectively perform NER tasks.

Fine-Grained Mechanical Chinese Named Entity Recognition Based on ALBERT-AttBiLSTM-CRF and Transfer Learning

Manufacturing text often exists as unlabeled data; the entity is fine-grained and the extraction is difficult. The above problems mean that the manufacturing industry knowledge utilization rate is low. This paper proposes a novel Chinese fine-grained NER (named entity recognition) method based on symmetry lightweight deep multinetwork collaboration (ALBERT-AttBiLSTM-CRF) and model transfer considering active learning (MTAL) to research fine-grained named entity recognition of a few labeled Chinese textual data types. The method is divided into two stages. In the first stage, the ALBERT-AttBiLSTM-CRF was applied for verification in the CLUENER2020 dataset (Public dataset) to get a pretrained model; the experiments show that the model obtains an F1 score of 0.8962, which is better than the best baseline algorithm, an improvement of 9.2%. In the second stage, the pretrained model was transferred into the Manufacturing-NER dataset (our dataset), and we used the active learning strategy to optimize the model effect. The final F1 result of Manufacturing-NER was 0.8931 after the model transfer (it was higher than 0.8576 before the model transfer); so, this method represents an improvement of 3.55%. Our method effectively transfers the existing knowledge from public source data to scientific target data, solving the problem of named entity recognition with scarce labeled domain data, and proves its effectiveness.

Innovative Deep Neural Network Modeling for Fine-Grained Chinese Entity Recognition

Named entity recognition (NER) is a basic but crucial task in the field of natural language processing (NLP) and big data analysis. The recognition of named entities based on Chinese is more complicated and difficult than English, which makes the task of NER in Chinese more challenging. In particular, fine-grained named entity recognition is more challenging than traditional named entity recognition tasks, mainly because fine-grained tasks have higher requirements for the ability of automatic feature extraction and information representation of deep neural models. In this paper, we propose an innovative neural network model named En2BiLSTM-CRF to improve the effect of fine-grained Chinese entity recognition tasks. This proposed model including the initial encoding layer, the enhanced encoding layer, and the decoding layer combines the advantages of pre-training model encoding, dual bidirectional long short-term memory (BiLSTM) networks, and a residual connection mechanism. Hence, it can encode information multiple times and extract contextual features hierarchically. We conducted sufficient experiments on two representative datasets using multiple important metrics and compared them with other advanced baselines. We present promising results showing that our proposed En2BiLSTM-CRF has better performance as well as better generalization ability in both fine-grained and coarse-grained Chinese entity recognition tasks.

Fine Grained Named Entity Recognition via Seq2seq Framework

Fine-grained Named entity recognition (NER) is crucial to natural language processing (NLP) applications like relation extraction and knowledge graph construction. Most existing fine-grained NER systems suffer from inefficiency problem as they use manually annotated training datasets. To address such issue, our NER system could automatically generate datasets from Wikipedia in distant supervision paradigm through mapping hyperlinks in Wikipedia documents to Freebase. In addition, previous NER models can not effectively process fine-grained labels with more than 100 types. So we introduce a ‘BIO’ tagging strategy which can identify the position and type attributes simultaneously. Such tagging scheme transfers NER problem into a sequence-to-sequence (seq2seq) based issue. We propose a seq2seq framework to comprehend the input sentence in a comprehensive way. Specifically, we adopt a Bi-LSTM as the encoder to equally process the past and future information of the input. Then we add a self-attention mechanism to handle the long-term dependency problem in a long sequence. When classifying the entity tags, we choose CRF model as it adds more constraints to avoid position logical problem. Experiments are performed on large-scale datasets for fine-grained NER tasks. Experimental results verify the effectiveness of FSeqC, and it outperforms other state-of-the-art alternatives consistently and significantly.

Fine-Grained Named Entity Recognition in Question Answering with DBpedia

Named Entity Recognition in Question Answer tasks can help the language understanding. Traditional NER task usually solved by a supervised model, which need a large number of annotated corpora and extract context information as feature for training, however questions are usually short sentences, with little context information, little public annotated corpora. On the other hand coarse-grained named entities’ types cannot offer enough information for question understanding. In this paper, we propose a novel model to address both problems, using a distant supervised method. Firstly, we use the web search to obtain more relevant information. Secondly, we present a greedy n-grams algorithm to extract the entity mentions. Finally, we use the kNN to classification and get fine-gained entity types combining with the entity mentions which can link with DBpedia. Experimental results show that our model outperforms various state-of-art systems in public dataset--TREC.