Entity Recognition Approach Research Articles

A Transfer Learning-Based Pairwise Information Extraction Framework Using BERT and Korean-Language Modification Relationships

Most named entity recognition approaches employing BERT-based transfer learning focus solely on extracting independent and simple tags, neglecting the sequence and dependency features inherent in the named-entity tags. Consequently, these basic BERT-based methods fall short in domains requiring the extraction of more intricate information, such as the detailed characteristics of products, services, and places from user reviews. In this paper, we introduce an end-to-end information extraction framework comprising three key components: (1) a tagging scheme that effectively represents detailed characteristics; (2) a BERT-based transfer learning model designed for extracting named-entity tags, utilizing both general linguistic features learned from a large corpus and the sequence and symmetric-dependency features of the named-entity tags; and (3) a pairwise information extraction algorithm that pairs features with their corresponding symmetric modifying words to extract detailed information.

A Named Entity Recognition Approach for Electronic Medical Records Using BERT Semantic Enhancement and BiLSTM

Aiming at the problems of missing local context features, single word vector representation, and low entity recognition accuracy, a method for e-medical recording with named entity recognition, which is based on BERT and model fusion, is proposed. First, with the model of BERT for pre-training, the preceding and following contextual information is fused for the enhancement of word semantic representation and alleviation of the problem of polysemy; second, the network of bi-directional long-short term memory is for obtaining the sequence feature matrix, generation of optimal sequence in global sense achieved through the conditional random field model; finally, data enhancement is used to alleviate the class imbalance and improve the model ability in generalization. Results of the experiments find model proposal measured by F1 on CCKS21 data set reaches 0.8548, which is 0.51% and 0.08% higher than models with ID-CNNs-CRF and multi-task RNN. This demonstrates the excellent performance of the method proposed in this paper in improving named entity recognition.

Transformer-based approach to variable typing

The upsurge of multifarious endeavors across scientific fields propelled Big Data in the scientific domain. Despite the advancements in management systems, researchers find that mathematical knowledge remains one of the most challenging to manage due to the latter's inherent heterogeneity. One novel recourse being explored is variable typing where current works remain preliminary and, thus, provide a wide room for contribution. In this study, a primordial attempt to implement the end-to-end Entity Recognition (ER) and Relation Extraction (RE) approach to variable typing was made using the BERT (Bidirectional Encoder Representations from Transformers) model. A micro-dataset was developed for this process. According to our findings, the ER model and RE model, respectively, have Precision of 0.8142 and 0.4919, Recall of 0.7816 and 0.6030, and F1-Scores of 0.7975 and 0.5418. Despite the limited dataset, the models performed at par with values in the literature. This work also discusses the factors affecting this BERT-based approach, giving rise to suggestions for future implementations.

Medical Named Entity Recognition Fusing Part-of-Speech and Stroke Features

It is highly significant from a research standpoint and a valuable practice to identify diseases, symptoms, drugs, examinations, and other medical entities in medical text data to support knowledge maps, question and answer systems, and other downstream tasks that can provide the public with knowledgeable answers. However, when contrasted with other languages like English, Chinese words lack a distinct dividing line, and medical entities have problems such as long length and multiple entity types nesting. Therefore, to address these issues, this study suggests a medical named entity recognition (NER) approach that combines part-of-speech and stroke features. First, the text is fed into the BERT pre-training model to get the semantic representation of the text, while the part-of-speech feature vector is obtained using the part-of-speech dictionary, and the stroke feature of the text is extracted through a convolution neural network (CNN). The word vector is then joined with the part-of-speech and stroke feature vectors, respectively, and input into the BiLSTM and CRF layer for training. Additionally, to balance the disparity in data volume across several types of entities, the class-weighted loss function is included in the loss function. According to the experimental findings, our model’s F1 score on the CCKS2019 dataset reaches 78.65%, and the recognition performance exceeds many existing algorithms.

A Chinese nested named entity recognition approach using sequence labeling

PurposeThis study aims to introduce an innovative approach that uses a decoder with multiple layers to accurately identify Chinese nested entities across various nesting depths. To address potential human intervention, an advanced optimization algorithm is used to fine-tune the decoder based on the depth of nested entities present in the data set. With this approach, this study achieves remarkable performance in recognizing Chinese nested entities.Design/methodology/approachThis study provides a framework for Chinese nested named entity recognition (NER) based on sequence labeling methods. Similar to existing approaches, the framework uses an advanced pre-training model as the backbone to extract semantic features from the text. Then a decoder comprising multiple conditional random field (CRF) algorithms is used to learn the associations between granularity labels. To minimize the need for manual intervention, the Jaya algorithm is used to optimize the number of CRF layers. Experimental results validate the effectiveness of the proposed approach, demonstrating its superior performance on both Chinese nested NER and flat NER tasks.FindingsThe experimental findings illustrate that the proposed methodology can achieve a remarkable 4.32% advancement in nested NER performance on the People’s Daily corpus compared to existing models.Originality/valueThis study explores a Chinese NER methodology based on the sequence labeling ideology for recognizing sophisticated Chinese nested entities with remarkable accuracy.

Indonesian news classification application with named entity recognition approach

Nowadays, many netizens search for news via search engines with countless amounts of information, so it is increasingly difficult to determine when the number of news articles that appear changes very quickly and dynamically. Thus, it is necessary to process the extraction of news information to display the core information of the news. Problems arise, especially in Indonesian, which has a structure of various noun phrase entities with shallow parsing or grammatical induction. Named Entity Recognition (NER) has the opportunity to overcome this because it can extract news entities in depth, starting from proper nouns in text documents containing information search, machine translation, answering questions, and automatic summarization. This study aims to apply NER in Indonesian language news classification. This study uses Design-Based Research whose process includes (1) pre-implementation, (2) design, (3) implementation and revision, and finally, (4) reflection and evaluation. This application was developed on the platform python, streamlit, BeautifulSoup, gnews, and spacy library. The results of application accuracy testing have an F1-score value of 89.69% for all entities consisting of place, figure, day, date, and organization.

Text and Dynamic Network Analysis for Measuring Technological Convergence: A Case Study on Defense Patent Data

Identifying technology convergence is getting harder, especially in fast-evolving and cross-technological fields. On the other side, natural language processing and network analysis researchers is to provide a novel method for mapping technologies and their relations over time, in order to identify dynamic patterns of convergence and to test it on the C4ISTAR field, a defense-related cross-technological field. The methodology automatically extracts technologies from a corpus of scientific papers, policy documents, websites, and company reports using Named Entity Recognition approaches. These technologies are then retrieved from the second corpus of more than 300 thousand patents related to the C4ISTAR domain, measuring their cooccurrences over time. Finally, using the time-varying network analysis, we were able to identify and measure the pattern of technological convergence.

Medical Name Entity Recognition Based on Lexical Enhancement and Global Pointer

Named entity recognition (NER) in biological sources, also called medical named entity recognition (MNER), attempts to identify and categorize medical terminology in electronic records. Deep neural networks have recently demonstrated substantial effectiveness in MNER. However, Chinese MNER has issues that cannot use lexical information and involve nested entities. To address these problems, we propose a model which can handle both nested and non-nested entities. The model uses a simple lexical enhancement method for merging lexical information into each character's vector representation, and then uses the Global Pointer approach for entity recognition. Furthermore, we retrain a pre-trained model with a Chinese medical corpus to incorporate medical knowledge, resulting in F1 score of 68.13% on the nested dataset CMeEE, 95.56% on the non-nested dataset CCKS2017, 85.89% on CCKS2019, and 92.08% on CCKS2020. These data demonstrate the efficacy of our proposed model.

Road-Related Information Mining from Social Media Data: A Joint Relation Extraction and Entity Recognition Approach

Social media data have been gradually regarded as a prospective social sensor in the transportation domain for capturing road conditions. Most existing social media data-based sensors (SMDbSs) of road conditions, however, rely heavily on lexicon-based methods for information extraction and provide coarse-grained location information. Hence, this work newly devises an SMDbS based on joint relation extraction and entity recognition for sensing road conditions from social media data, which eliminates the reliance on lexicon-based methods and offers finer-grained location information in comparison with existing SMDbSs. This SMDbS development consists of four major steps, including data collection and annotation, data cleansing, two-stage information extraction, and model verification. A tweet dataset in Lexington city is exploited to demonstrate this SMDbS, which shows satisfactory information extraction performance. This study would help facilitate social media data to be an extra information source in the transportation domain.

Token Relation Aware Chinese Named Entity Recognition

Due to the lack of natural delimiters, most Chinese Named Entity Recognition (NER) approaches are character-based and utilize an external lexicon to leverage the word-level information. Although they have achieved promising results, the latent words they introduced are still non-contextualized. In this paper, we investigate three relations, i.e., adjacent relation between characters, character co-occurrence relation between latent words, and dependency relation among tokens, to address this issue. Specifically, we first establish the local context for latent words and then propose a masked self-attention mechanism to incorporate such local contextual information. Besides, since introducing external knowledge such as lexicon and dependency relation inevitably brings in some noises, we propose a gated information controller to handle this problem. Extensive experimental results show that the proposed approach surpasses most similar methods on public datasets and demonstrates its promising potential.

A Named Entity and Relationship Extraction Method from Trouble-Shooting Documents in Korean

In enterprises operating large-scale equipment, such as plants, maintenance workers must quickly and accurately find and understand the information in the equipment maintenance documents to perform maintenance tasks effectively. If the equipment maintenance documents include sentences with semantically ambiguous expressions, it will interfere with the maintenance knowledge search, and it may affect the maintenance performance of engineers. In order to solve these problems, text-based research of maintenance documents have been done to extract the key information or knowledge from these documents. Previous studies focused on finding the technical terminologies or calculating the similarity of documents using named entity recognition approaches. This paper proposes a method to extract knowledge of not only the technical terminologies but also their relations. The proposed method uses a rule-based approach that can be applied to the results of a named entity recognition approach and a dependency parsing approach. The named entity recognition approach found technical terms and the dependency parsing approach provided sentence structure information, so that the proposed method showed that a set of rules can extract maintenance knowledge, including entities and their relations. Trouble-shooting documents in the field were used as an experiment to demonstrate the effectiveness of the proposed method, and the experiment showed the possibility of practical use of the proposed method.

An improved entity recognition approach to cyber-social knowledge provision of intellectual property using a CRF-LSTM model

• An improved deep learning model using CRF and Bi-LSTM is given to achieve named entity recognition with unbalanced labels. • An adaptive G-Score is used to compare the fitting ability of models by evaluating the gap between Precision and Recall. • A multiclassification cross-entropy loss function has been presented for training with additional weights. With the development of cutting-edge IT technologies, e.g. Big Data, Knowledge Engineering , etc., traditional Intellectual Property (IP) services have depicted high redundancy and low efficiency during management of such large-scale of data. Recent advancement of Artificial Intelligence (AI) and Deep Learning (DL) models has been accelerating relevant research activities being investigated on Knowledge Graph (KG) schemes and applications in different domains, such as medical services, social media, etc. However, when IP services and their cyber-social provision are taken into account, relevant approaches suffer from unbalanced labels against training results, and inappropriate evaluation metrics not well reflecting the impact of the unbalance. In this paper, a deep learning model combining Conditional Random Field and Bidirectional LSTM has been proposed, in order to achieve named entity recognition with unbalanced labels. An adaptive metric, G-Score was introduced to compare the fitting ability of models by evaluating the gap between Precision and Recall. According to the results, the proposed model can effectively recognize the potential named entities with outperformance over other relevant models.

CovidPubGraph: A FAIR Knowledge Graph of COVID-19 Publications

The rapid generation of large amounts of information about the coronavirus SARS-CoV-2 and the disease COVID-19 makes it increasingly difficult to gain a comprehensive overview of current insights related to the disease. With this work, we aim to support the rapid access to a comprehensive data source on COVID-19 targeted especially at researchers. Our knowledge graph, CovidPubGraph, an RDF knowledge graph of scientific publications, abides by the Linked Data and FAIR principles. The base dataset for the extraction is CORD-19, a dataset of COVID-19-related publications, which is updated regularly. Consequently, CovidPubGraph is updated biweekly. Our generation pipeline applies named entity recognition, entity linking and link discovery approaches to the original data. The current version of CovidPubGraph contains 268,108,670 triples and is linked to 9 other datasets by over 1 million links. In our use case studies, we demonstrate the usefulness of our knowledge graph for different applications. CovidPubGraph is publicly available under the Creative Commons Attribution 4.0 International license.

“FIJO”: a French Insurance Soft Skill Detection Dataset

Understanding the evolution of job requirements is becoming more important for workers, companies and public organizations to follow the fast transformation of the employment market. Fortunately, recent natural language processing (NLP) approaches allow for the development of methods to automatically extract information from job ads and recognize skills more precisely. However, these efficient approaches need a large amount of annotated data from the studied domain which is difficult to access, mainly due to intellectual property. This article proposes a new public dataset, FIJO, containing insurance job offers, including many soft skill annotations. To understand the potential of this dataset, we detail some characteristics and some limitations. Then, we present the results of skill detection algorithms using a named entity recognition approach and show that transformers-based models have good token-wise performances on this dataset. Lastly, we analyze some errors made by our best model to emphasize the difficulties that may arise when applying NLP approaches.

The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records

BackgroundAffective characteristics are associated with depression severity, course, and prognosis. Patients’ affect captured by clinicians during sessions may provide a rich source of information that more naturally aligns with the depression course and patient-desired depression outcomes.ObjectiveIn this paper, we propose an information extraction vocabulary used to pilot the feasibility and reliability of identifying clinician-recorded patient affective states in clinical notes from electronic health records.MethodsAffect and mood were annotated in 147 clinical notes of 109 patients by 2 independent coders across 3 pilots. Intercoder discrepancies were settled by a third coder. This reference annotation set was used to test a proof-of-concept natural language processing (NLP) system using a named entity recognition approach.ResultsConcepts were frequently addressed in templated format and free text in clinical notes. Annotated data demonstrated that affective characteristics were identified in 87.8% (129/147) of the notes, while mood was identified in 97.3% (143/147) of the notes. The intercoder reliability was consistently good across the pilots (interannotator agreement [IAA] >70%). The final NLP system showed good reliability with the final reference annotation set (mood IAA=85.8%; affect IAA=80.9%).ConclusionsAffect and mood can be reliably identified in clinician reports and are good targets for NLP. We discuss several next steps to expand on this proof of concept and the value of this research for depression clinical research.

Detecting a citizens' activity profile of an urban territory through natural language processing of social media data

The article presents the premises, process, and outcomes of the research, devoted to investigation of the suitability of natural language processing approaches (named entity recognition and subject-predicate-object triplets’ extraction, in particular), applied to social media data, for the problem of building a profile of citizens' activity in an urban territory. Using the named entity recognition approach, supplemented with the custom method of named urban entities distillation, it was possible to build a detailed and representative list of named urban entities for the sample territory of Hatfield, Hertfordshire. Using the subject-predicate-object triplets’ extraction approach, supplemented with the custom activity description patterns, it was possible to get the picture of citizens’ activity corresponding to the identified urban entities. The outcomes were verified on the Twitter and Instagram social networks data and evaluated from the perspectives of the resulting profile quality.

Digging in Documents: Using Text Mining to Reveal Hidden Knowledge in the Lithium-Ion Battery Literature

Scientific publications constitute the main information source of battery-related data for academic research. There is an impressive number of papers published on Lithium Ion Batteries (LIBs) for instance. Such impressive amount of data in principle could be used by machine learning (ML) methods in order to optimize efforts and even make possible the discovery of new optimal experimental conditions. Yet, the majority is published as unstructured data, making ML analysis difficult. Here, in order to unravel researchers’ habits when reporting their results, we present a text mining study using primarily publications containing original experimental data from the LIB field. We analyze the reported data structure and we discuss the remaining challenges for applying automated knowledge discovery techniques. We then present a Named Entity Recognition approach to automatically extract quantified parameters from the battery literature and how to build an in house generated database containing a wide diversity of materials, experimental parameters and conditions. Our in house approach achieves a data extraction F-scores ≥ 90% and can thereby pave the way towards an automatized way of generating large databases for use in battery ML models. References El-Bouysidy, H., Lombardo, T., Primo, E. N., Duquesnoy, M., Morcrette, M., Johansson, P., Simon, P., Grimaud, A., & Franco, A. A. submitted (2020).Torayev, Amangeldi, Pieter CMM Magusin, Clare P. Grey, Céline Merlet, and Alejandro A. Franco. Journal of Physics: Materials 2, no. 4 (2019): 044004.

An Automatic Participant Detection Framework for Event Tracking on Twitter

Topic Detection and Tracking (TDT) on Twitter emulates human identifying developments in events from a stream of tweets, but while event participants are important for humans to understand what happens during events, machines have no knowledge of them. Our evaluation on football matches and basketball games shows that identifying event participants from tweets is a difficult problem exacerbated by Twitter’s noise and bias. As a result, traditional Named Entity Recognition (NER) approaches struggle to identify participants from the pre-event Twitter stream. To overcome these challenges, we describe Automatic Participant Detection (APD) to detect an event’s participants before the event starts and improve the machine understanding of events. We propose a six-step framework to identify participants and present our implementation, which combines information from Twitter’s pre-event stream and Wikipedia. In spite of the difficulties associated with Twitter and NER in the challenging context of events, our approach manages to restrict noise and consistently detects the majority of the participants. By empowering machines with some of the knowledge that humans have about events, APD lays the foundation not just for improved TDT systems, but also for a future where machines can model and mine events for themselves.

Named entity recognition for extracting concept in ontology building on Indonesian language using end-to-end bidirectional long short term memory

Information Extraction has been widely used to extract information from text. Named Entity Recognition (NER) is one of the primary tasks of Information Extraction to extract entities such as person, location, and organization. Extraction from text collection is essential to obtain information from unstructured text. Moreover, Named Entity Recognition is part of ontology building, which is the main objective of this research. Ontology can be built on the basis of a collection of concepts and relation between concepts. Concepts in ontology usually consist of a group of entities and are obtained using Noun Phrase Extraction or Named Entity Recognition. Our main focus in this research is to extract concepts in Ontology Building automatically using Named Entity Recognition. In this paper, Named Entity Recognition was chosen as our approach due to the lack of results from the previous Noun Phrase Extraction works, which is not all nouns obtained are entities. Our proposed methodology for Named Entity Recognition is applying an end-to-end model using Bidirectional Long Short Term Memory (Bi-LSTM). Bi-LSTM is able to perform a sequence classification task by understanding the context of the input. Named Entity Recognition approaches in the previous study uses Part-of-Speech (POS) Tagging in the preprocessing phase by using other tools or models. This Part-of Speech is also used as a feature to improve the performance of Named Entity Recognition. Our proposed methodology provides an end-to-end system that can be used for both POS Tagging and Named Entity Recognition. By using our proposed end-to-end model, no additional tool is needed for Part-of-Speech Tagging. This the advantage of our model compared to other models. Experiments were conducted on news documents that were labeled with four types of entity classes and 35 types of part-of-speech. The target entities that we have extracted in this study are person, location, organization, and miscellaneous. We evaluated the performance of our model using F1-Score. We have achieved the best F1-Score for Part-of-Speech Tagging of 91.79% and Named Entity Recognition of 83.18%.

Computational Reproducibility of Named Entity Recognition methods in the biomedical domain

Unsupervised Named Entity Recognition (NER) approaches do not depend on labelled data to function properly but rather on a source of knowledge, in which promising candidates can be looked up to find the corresponding concept. In the biomedical domain knowledge source like this already exists; namely the Unified Medical Language System (UMLS). In this paper, three different unsupervised NER models using UMLS, namely MetaMap, cTakes and MetaMapLite are evaluated and compared from the results published by Demner-Fushman, Rogers and Aronson (2017) and Reategui and Ratte (2018). The Unsupervised Biomedical Named Entity Recognition framework (UB-NER) is developed, with which the results of the experiments of the three models, five datasets and two NER tasks are presented.