Named Entity Recognition Research Articles

BVTED: A Specialized Bilingual (Chinese–English) Dataset for Vulnerability Triple Extraction Tasks

Extracting knowledge from cyber threat intelligence is essential for understanding cyber threats and implementing proactive defense measures. However, there is a lack of open datasets in the Chinese cybersecurity field that support both entity and relation extraction tasks. This paper addresses this gap by analyzing vulnerability description texts, which are standardized and knowledge-dense, to create a vulnerability knowledge ontology comprising 13 entities and 15 relations. We annotated 27,311 unique vulnerability description sentences from the China National Vulnerability Database, resulting in a dataset named BVTED for cybersecurity knowledge triple extraction tasks. BVTED contains 97,391 entities and 69,614 relations, with entities expressed in a mix of Chinese and English. To evaluate the dataset’s value, we trained five deep learning-based named entity recognition models, two relation extraction models, and two joint entity–relation extraction models on BVTED. Experimental results demonstrate that models trained on this dataset achieve excellent performance in vulnerability knowledge extraction tasks. This work enhances the extraction of cybersecurity knowledge triples from mixed Chinese and English threat intelligence corpora by providing a comprehensive ontology and a new dataset, significantly aiding in the mining, analysis and utilization of the knowledge embedded in cyber threat intelligence.

Biomedical named entity recognition through improved balanced undersampling for addressing class imbalance and preserving contextual information

Biomedical named entity recognition through improved balanced undersampling for addressing class imbalance and preserving contextual information

APT-scope: A novel framework to predict advanced persistent threat groups from enriched heterogeneous information network of cyber threat intelligence

Addressing the expanding Advanced Persistent Threat (APT) landscape is crucial for governments, enterprises and threat intelligence research groups. While defenders often rely on tabular formats for assets like logs, alerts, firewall rules; attackers leverage a graph-based mindset. In this work, we propose a novel multistage framework named APT-Scope which employs a comprehensive approach to Cyber Threat Intelligence (CTI) analysis on qualified real-world data. APT-Scope workflow consists of data gathering, enrichment, and analysis stages, where relationships between entities are used to construct a Heterogeneous Information Network (HIN). We applied CTI enrichment using additional active data collection techniques like DNS and Whois lookups, port scans, SSL footprinting, named entity recognition via SpaCy, and constructed a machine learning pipeline to predict relationships between entities using FastRP and Logistic Regression. By analyzing the resulting HIN, we discovered aliases for APT groups and predicted threat actors of APT attacks with unknown perpetrators. We observed AUCPR metrics as train score = 96.57% and test score = 92.36%. Our work is beneficial to oversee the entire APT landscape, steer ongoing and future CTI operations and make strategic decisions.

The interactive fusion of characters and lexical information for Chinese named entity recognition

Many studies have demonstrated that incorporating lexical information into characters can effectively improve the performance of Chinese Named Entity Recognition (CNER). However, we argue that previous studies have not extensively explored the interactive relationship between characters and lexical information, and have only used the lexical information to enhance character-level representation. To address this limitation, we propose an interactive fusion approach that integrates characters and lexical information for CNER. Specifically, we first design graph attention networks to initially fuse character and lexical information within an interactive graph structure. Additionally, by introducing methods such as feedforward neural networks, residual connections, and layer normalization, the fusion effect of the graph attention network is further enhanced. Finally, concatenating and reducing dimensionality of character feature vectors and lexical feature vectors to achieve secondary fusion, thereby obtaining a more comprehensive feature representation. Experimental results on multiple datasets demonstrate that our proposed model outperforms other models that fuse lexical information. Particularly, on the CCKS2020 and Ontonotes datasets, our model achieves higher F1 scores than previous state-of-the-art models. The code is available via the link: https://github.com/wangye0523/The-interactive-fusion-of-characters-and-lexical-information-for-Chinese-named-entity-recognition.

Bootstrapping public entities. Domain-specific NER for public speakers

ABSTRACT Named Entity Recognition (NER) is a supervised machine learning task that finds various applications in automated content analysis, as the identification of entities is vital for understanding public discourse. However, sometimes the standard NER labels are not specific enough for a given domain. We introduce Public Entity Recognition (PER). PER is a domain-specific version of NER, that is trained for five entity types that are common to public discourse: politicians, parties, authorities, media, and journalists. PER can be used for pre-processing documents, in a pipeline with other classifiers or directly for analyzing information in texts. The taxonomy for PER is taken from the database of (German) public speakers and aims at low-threshold integration into computational social science research. We experiment with different training settings, involving weakly supervised training and training on manually annotated data. We evaluate multilingual transformer models of different sizes against rule-based entity matching and find that the models do not only outperform the baseline but also reach competitive absolute scores of around .8 and higher in F1. We further test for generalization and domain adaptation. We show that with only around 100–150 additional sentences, the model can be adapted to new languages.

A multi-task framework based on decomposition for multimodal named entity recognition

Given a text-image pair, Multimodal Named Entity Recognition (MNER) is the task of identifying and categorizing entities in the text. Most existing work performs named entity labeling directly using final token representations derived by fusing image and text representations. Although they achieve promising results, these work may fail to effectively exploit text and image modalities. This is because they neglect the difference in the role of the two modalities: text modality can detect the boundary of an entity, while image modality is introduced to disambiguate the category of the entity. Based on these findings, in this paper, we construct two auxiliary tasks based on the decomposition strategy and propose a multi-task framework for MNER. Specifically, we first decompose MNER into two auxiliary tasks: entity boundary detection task and entity category classification task. Here, the former treats only the text modality as input and outputs the boundary labels, since it can achieve satisfactory boundary results by itself. The latter uses two modalities to yield category labels where image modality is dedicated to disambiguating categories. These two auxiliary tasks allow the effective exploitation of text and image modalities and put them back into their respective roles. Then, we vectorize their results to improve entity recognition using label clues from auxiliary tasks. Finally, we fuse features from text and image modalities and label embeddings from auxiliary tasks to fulfill MNER. Experimental results on two widely used MNER datasets show that our framework can yield new SOTA performance.

Camouflaged named entity recognition in 2D sentence representation

Organizing all spans of a sentence into a two-dimensional (2D) representation unfolds a semantic plane. It has the advantage to resolve nested semantic structures and to build linguistic dependencies across a whole sentence. The main problem is that neighboring elements in the semantic plane are span representations referred to overlapped phrases. Because these representations share the same contextual features in a sentence, a true entity representation is early faded into the background surroundings. It leads to camouflaged named entities in the 2D sentence representation. In this paper, we propose a finer-scale and coarse-scale sentence representation to support camouflaged named entity recognition. The mixed-scale representation has the ability to encode differential clues between entity representations. It is effective to distinguish entity representations from the background surroundings for recognizing camouflaged named entities. Compared with the state-of-the-art models, the results show that our method achieves competitive performance on five public datasets.

Zero-shot evaluation of ChatGPT for food named-entity recognition and linking.

Recognizing and extracting key information from textual data plays an important role in intelligent systems by maintaining up-to-date knowledge, reinforcing informed decision-making, question-answering, and more. It is especially apparent in the food domain, where critical information guides the decisions of nutritionists and clinicians. The information extraction process involves two natural language processing tasks named entity recognition-NER and named entity linking-NEL. With the emergence of large language models (LLMs), especially ChatGPT, many areas began incorporating its knowledge to reduce workloads or simplify tasks. In the field of food, however, we noticed an opportunity to involve ChatGPT in NER and NEL. To assess ChatGPT's capabilities, we have evaluated its two versions, ChatGPT-3.5 and ChatGPT-4, focusing on their performance across both NER and NEL tasks, emphasizing food-related data. To benchmark our results in the food domain, we also investigated its capabilities in a more broadly investigated biomedical domain. By evaluating its zero-shot capabilities, we were able to ascertain the strengths and weaknesses of the two versions of ChatGPT. Despite being able to show promising results in NER compared to other models. When tasked with linking entities to their identifiers from semantic models ChatGPT's effectiveness falls drastically. While the integration of ChatGPT holds potential across various fields, it is crucial to approach its use with caution, particularly in relying on its responses for critical decisions in food and bio-medicine.

AI-Driven Thoracic X-ray Diagnostics: Transformative Transfer Learning for Clinical Validation in Pulmonary Radiography.

Our research evaluates advanced artificial (AI) methodologies to enhance diagnostic accuracy in pulmonary radiography. Utilizing DenseNet121 and ResNet50, we analyzed 108,948 chest X-ray images from 32,717 patients and DenseNet121 achieved an area under the curve (AUC) of 94% in identifying the conditions of pneumothorax and oedema. The model's performance surpassed that of expert radiologists, though further improvements are necessary for diagnosing complex conditions such as emphysema, effusion, and hernia. Clinical validation integrating Latent Dirichlet Allocation (LDA) and Named Entity Recognition (NER) demonstrated the potential of natural language processing (NLP) in clinical workflows. The NER system achieved a precision of 92% and a recall of 88%. Sentiment analysis using DistilBERT provided a nuanced understanding of clinical notes, which is essential for refining diagnostic decisions. XGBoost and SHapley Additive exPlanations (SHAP) enhanced feature extraction and model interpretability. Local Interpretable Model-agnostic Explanations (LIME) and occlusion sensitivity analysis further enriched transparency, enabling healthcare providers to trust AI predictions. These AI techniques reduced processing times by 60% and annotation errors by 75%, setting a new benchmark for efficiency in thoracic diagnostics. The research explored the transformative potential of AI in medical imaging, advancing traditional diagnostics and accelerating medical evaluations in clinical settings.

A data fusion method for maritime traffic surveillance: The fusion of AIS data and VHF speech information

The more comprehensive information supply for Vessel Traffic Service (VTS), the greater capacity for maritime traffic surveillance. To supply ship intention information contained in very high frequency (VHF) speech for VTS, we propose an automatic identification system (AIS) data and VHF speech information fusion method. In the first part of this method, the feature information of VHF speech is extracted by Bert-BiLSTM-CRF-based entity extraction method, likes location, ship name, and ship intention information, etc. In the second part, we design a spatio-temporal range to filter AIS data, the temporal range is determined by receive time of VHF speech, and the spatial range is determined by the location information extracted from VHF speech. In the third part, by measuring the ship name similarity between AIS data and VHF speech, the barriers to fusion are successfully overcome. The similarity is calculated by combining with Chinese pinyin character similarity and Arabic number similarity and Chinese pinyin characters are encoded by pronunciation characters, calculated by the dynamic time warping (DTW) distance. Through experiments, the method can effectively provide comprehensive information for maritime traffic surveillance, and even when crews misreport ship name within a small deviation, our method still possesses a great fusion capability.

Indonesian disaster named entity recognition from multi source information using bidirectional LSTM (BiLSTM)

Precise logistic support is essential after a disaster occurs. It must be timely, accurate, targeted, and based on existing needs. However, obtaining sufficient and accurate information related to logistic distribution locations remains a key problem. Therefore, implementing Named Entity Recognition (NER) can address this issue. In recent years, news coverage through Indonesian digital news media and social media accounts has emerged as a promising source for building a disaster data corpus. This study implemented NER to extract and identify named entities from text-based information, particularly from Indonesian digital news media. In addition to using regular entities from the NER standard, this study introduced new entities specialized for disaster-related information, including DISASTER, SCALE, SUPPLIES, CASUALTIES, and OUTSIDE. The new disaster corpus in the Indonesian language for the NER model was obtained with an imbalanced dataset composition. To overcome this problem, random oversampling was applied. This study also utilized the BiLSTM model to recognize each entity in new textual information, evaluating its performance when the proposed Indonesian disaster corpus was used as a training reference in the deep learning model. Several optimization algorithms applied in BiLSTM were evaluated. The results showed improved BiLSTM performance using Adam optimization and a balanced corpus. Performance indicators achieved were 93.4 %, 82.4 %, and 87.5 % for precision, recall, and F1-score, respectively. The BiLSTM network captured long-range dependencies in sequential data provided by NER. Oversampling ensured that the proposed NER model could precisely recognize all entities and reduce biased results. Thus, the BiLSTM method can better identify entities in the textual corpus of Indonesian disaster-related online news.

Revisiting named entity recognition in food computing: enhancing performance and robustness

In the ever-evolving domain of food computing, named entity recognition (NER) presents transformative potential that extends far beyond mere word tagging in recipes. Its implications encompass intelligent recipe recommendations, health analysis, and personalization. Nevertheless, existing NER models in food computing encounter challenges stemming from variations in recipe input standards, limited annotations, and dataset quality. This article addresses the specific problem of ingredient NER and introduces two innovative models: SINERA, an efficient and robust model, and SINERAS, a semi-supervised variant that leverages a Gaussian Mixture Model (GMM) to learn from untagged ingredient list entries. To mitigate issues associated with data quality and availability in food computing, we introduce the ARTI dataset, a diverse and comprehensive repository of ingredient lines. Additionally, we identify and tackle a pervasive challenge—spurious correlations between entity positions and predictions. To address this, we propose a set of data augmentation rules tailored for food NER. Extensive evaluations conducted on the ARTI dataset and a revised TASTEset dataset underscore the performance of our models. They outperform several state-of-the-art benchmarks and rival the BERT model while maintaining smaller parameter sizes and reduced training times.

Segmentation-aware relational graph convolutional network with multi-layer CRF for nested named entity recognition

AbstractNamed Entity Recognition (NER) is fundamental in natural language processing, involving identifying entity spans and types within a sentence. Nested NER contains other entities, which pose a significant challenge, especially pronounced in the domain of medical-named entities due to intricate nesting patterns inherent in medical terminology. Existing studies can not capture interdependencies among different entity categories, resulting in inadequate performance in nested NER tasks. To address this problem, we propose a novel Layer-based architecture with Segmentation-aware Relational Graph Convolutional Network (LSRGCN) for Nested NER in the medical domain. LSRGCN comprises two key modules: a shared segmentation-aware encoder and a multi-layer conditional random field decoder. The former part provides token representation including boundary information from sentence segmentation. The latter part can learn the connections between different entity classes and improve recognition accuracy through secondary decoding. We conduct experiments on four datasets. Experimental results demonstrate the effectiveness of our model. Additionally, extensive studies are conducted to enhance our understanding of the model and its capabilities.

Parental report of vocabulary in 3- to 6-year-old Polish children: Reliable but not valid.

For over 30 years, parental reports have been used to study the vocabulary of children under 4 years of age. Research exploring parental checklists as a measure of vocabulary in older children is very limited. Typically, authors of parental checklists report the reliability of the developed tools but do not explore validity in terms of the agreement between parental assessments and the children's actual word knowledge. We aimed to explore the reliability and validity of a parental checklist for assessing vocabulary in children aged between 3 and 6 years. Furthermore, we aimed to evaluate the agreement between indirect (parental checklist) and direct (picture naming and picture recognition tasks) assessments of children's vocabulary. A group of 94 typically developing monolingual Polish-speaking children aged between 3 and 6 years were first directly tested onsite with picture naming and picture recognition tasks (Cross-Linguistic Lexical Tasks). Subsequently, the participants' parents completed an online checklist containing the same set of 128 items and marked all the words that they had ever heard in their child's spontaneous speech. The parental checklist demonstrated very high internal consistency. The scores of the parental checklist and vocabulary tasks were moderately correlated. We compared the total number of words marked by parents and the number of items correctly identified by children in the picture naming and picture recognition tasks. In picture naming, we found no difference between the children's scores and the number of words selected by parents. However, parents selected significantly fewer words than children correctly recognised in the picture recognition task. When data were analysed at the level of individual items (i.e., whether parents selected exactly the same items that children answered correctly), we found that the level of agreement was low. The level of agreement correlated negatively with the children's vocabulary; that is, the more words a child knew, the lower the agreement between the direct measure and the parental checklist. Parental checklists should be used with caution in children aged between 3 and 6 years, especially if the assessed children have a large vocabulary and if item analysis is planned. Such checklists may be of more use in younger children or in children with limited vocabulary. What is already known on the subject Parental checklists are commonly used to assess the vocabulary of children younger than 4 years of age. Previous research has indicated that parental checklists are reliable in terms of internal consistency and valid in terms of predictive and convergent validity. What this paper adds to the existing knowledge This study introduces a parental checklist designed for assessing the vocabulary of monolingual Polish-speaking children aged between 3 and 6 years. Statistical analyses reveal that while the parental checklist exhibits high reliability, and the scores on the checklist correlate with direct measures of vocabulary, the agreement between parental reports and direct vocabulary measures (i.e., validity) is notably low, particularly when examining individual test items. What are the clinical implications of this work? These findings underscore the importance of exercising caution when using parental vocabulary checklists with children aged between 3 and 6 years. These checklists can serve as a replacement for direct vocabulary tests only when the general/overall score is needed. However, when specific words are the subject of interest, parental reports may not be a valid measure.

The biomedical relationship corpus of the BioRED track at the BioCreative VIII challenge and workshop.

The automatic recognition of biomedical relationships is an important step in the semantic understanding of the information contained in the unstructured text of the published literature. The BioRED track at BioCreative VIII aimed to foster the development of such methods by providing the participants the BioRED-BC8 corpus, a collection of 1000 PubMed documents manually curated for diseases, gene/proteins, chemicals, cell lines, gene variants, and species, as well as pairwise relationships between them which are disease-gene, chemical-gene, disease-variant, gene-gene, chemical-disease, chemical-chemical, chemical-variant, and variant-variant. Furthermore, relationships are categorized into the following semantic categories: positive correlation, negative correlation, binding, conversion, drug interaction, comparison, cotreatment, and association. Unlike most of the previous publicly available corpora, all relationships are expressed at the document level as opposed to the sentence level, and as such, the entities are normalized to the corresponding concept identifiers of the standardized vocabularies, namely, diseases and chemicals are normalized to MeSH, genes (and proteins) to National Center for Biotechnology Information (NCBI) Gene, species to NCBI Taxonomy, cell lines to Cellosaurus, and gene/protein variants to Single Nucleotide Polymorphism Database. Finally, each annotated relationship is categorized as 'novel' depending on whether it is a novel finding or experimental verification in the publication it is expressed in. This distinction helps differentiate novel findings from other relationships in the same text that provides known facts and/or background knowledge. The BioRED-BC8 corpus uses the previous BioRED corpus of 600 PubMed articles as the training dataset and includes a set of newly published 400 articles to serve as the test data for the challenge. All test articles were manually annotated for the BioCreative VIII challenge by expert biocurators at the National Library of Medicine, using the original annotation guidelines, where each article is doubly annotated in a three-round annotation process until full agreement is reached between all curators. This manuscript details the characteristics of the BioRED-BC8 corpus as a critical resource for biomedical named entity recognition and relation extraction. Using this new resource, we have demonstrated advancements in biomedical text-mining algorithm development. Database URL: https://codalab.lisn.upsaclay.fr/competitions/16381.

A Novel Rational Medicine Use System Based on Domain Knowledge Graph

Medication errors, which could often be detected in advance, are a significant cause of patient deaths each year, highlighting the critical importance of medication safety. The rapid advancement of data analysis technologies has made intelligent medication assistance applications possible, and these applications rely heavily on medical knowledge graphs. However, current knowledge graph construction techniques are predominantly focused on general domains, leaving a gap in specialized fields, particularly in the medical domain for medication assistance. The specialized nature of medical knowledge and the distinct distribution of vocabulary between general and biomedical texts pose challenges. Applying general natural language processing techniques directly to the medical domain often results in lower accuracy due to the inadequate utilization of contextual semantics and entity information. To address these issues and enhance knowledge graph production, this paper proposes an optimized model for named entity recognition and relationship extraction in the Chinese medical domain. Key innovations include utilizing Medical Bidirectional Encoder Representations from Transformers (MCBERT) for character-level embeddings pre-trained on Chinese biomedical corpora, employing Bi-directional Gated Recurrent Unit (BiGRU) networks for extracting enriched contextual features, integrating a Conditional Random Field (CRF) layer for optimal label sequence output, using the Piecewise Convolutional Neural Network (PCNN) to capture comprehensive semantic information and fusing it with entity features for better classification accuracy, and implementing a microservices architecture for the medication assistance review system. These enhancements significantly improve the accuracy of entity relationship classification in Chinese medical texts. The model achieved good performance in recognizing most entity types, with an accuracy of 88.3%, a recall rate of 85.8%, and an F1 score of 87.0%. In the relationship extraction stage, the accuracy reached 85.7%, the recall rate 82.5%, and the F1 score 84.0%.

A New Chinese Named Entity Recognition Method for Pig Disease Domain Based on Lexicon-Enhanced BERT and Contrastive Learning

A New Chinese Named Entity Recognition Method for Pig Disease Domain Based on Lexicon-Enhanced BERT and Contrastive Learning

Evaluation on Network Social Media Named Entity Recognition Model Based on Active Learning

The medical security privacy and named entity recognition (NER) technology under the blockchain technology has been a hot topic in all walks of life. As a typical representative of medical security risks and NER, the NER model of online social media based on active learning has attracted worldwide attention. NER is an important part of natural language processing. Traditional recognition technology usually requires a lot of external information, and through the manual identification of its features, which costs a lot of time and energy. In order to solve the shortcomings of traditional recognition algorithms and the lack of feature extraction in network media NER, a new active learning model was introduced in this paper. In the information age, people are increasingly demanding a large amount of text information, and NER technology came into being. Its main function is to accurately identify important information from text and provide useful information for high-level work. The initial design of NER system is mainly based on the recognition of rules, so as to realize the recognition of named entities. However, in a complex network environment, it takes a lot of time and energy to establish rules without conflicts, and it has poor mobility. In recent years, with the continuous development of computer technology, the use of machine learning to actively learn the unknown information in the target area reduces the workload of manual annotation, thus realizing the active learning of large amounts of data. The research showed that the recognition accuracy under the traditional NER was low, and the information processing speed was slow; the accuracy rate of NER based on active learning was as high as 97%, and the speed of information processing had also been greatly improved, which had solved many problems under the traditional mode. User satisfaction could be as high as 95%, which showed that the latter had broad prospects. The progress of the new era cannot be separated from the support of new technologies. The research of this article has important guiding significance for medical security privacy and the application of NER under blockchain technology.

Large Language Models in Healthcare and Medical Domain: A Review

The deployment of large language models (LLMs) within the healthcare sector has sparked both enthusiasm and apprehension. These models exhibit the remarkable ability to provide proficient responses to free-text queries, demonstrating a nuanced understanding of professional medical knowledge. This comprehensive survey delves into the functionalities of existing LLMs designed for healthcare applications and elucidates the trajectory of their development, starting with traditional Pretrained Language Models (PLMs) and then moving to the present state of LLMs in the healthcare sector. First, we explore the potential of LLMs to amplify the efficiency and effectiveness of diverse healthcare applications, particularly focusing on clinical language understanding tasks. These tasks encompass a wide spectrum, ranging from named entity recognition and relation extraction to natural language inference, multimodal medical applications, document classification, and question-answering. Additionally, we conduct an extensive comparison of the most recent state-of-the-art LLMs in the healthcare domain, while also assessing the utilization of various open-source LLMs and highlighting their significance in healthcare applications. Furthermore, we present the essential performance metrics employed to evaluate LLMs in the biomedical domain, shedding light on their effectiveness and limitations. Finally, we summarize the prominent challenges and constraints faced by large language models in the healthcare sector by offering a holistic perspective on their potential benefits and shortcomings. This review provides a comprehensive exploration of the current landscape of LLMs in healthcare, addressing their role in transforming medical applications and the areas that warrant further research and development.

An improved data augmentation approach and its application in medical named entity recognition

Performing data augmentation in medical named entity recognition (NER) is crucial due to the unique challenges posed by this field. Medical data is characterized by high acquisition costs, specialized terminology, imbalanced distributions, and limited training resources. These factors make achieving high performance in medical NER particularly difficult. Data augmentation methods help to mitigate these issues by generating additional training samples, thus balancing data distribution, enriching the training dataset, and improving model generalization. This paper proposes two data augmentation methods—Contextual Random Replacement based on Word2Vec Augmentation (CRR) and Targeted Entity Random Replacement Augmentation (TER)—aimed at addressing the scarcity and imbalance of data in the medical domain. When combined with a deep learning-based Chinese NER model, these methods can significantly enhance performance and recognition accuracy under limited resources. Experimental results demonstrate that both augmentation methods effectively improve the recognition capability of medical named entities. Specifically, the BERT-BiLSTM-CRF model achieved the highest F1 score of 83.587%, representing a 1.49% increase over the baseline model. This validates the importance and effectiveness of data augmentation in medical NER.