Building Knowledge Graphs Research Articles

Named Entity Recognition for Crop Diseases and Pests Based on Gated Fusion Unit and Manhattan Attention

Named entity recognition (NER) is a crucial step in building knowledge graphs for crop diseases and pests. To enhance NER accuracy, we propose a new NER model—GatedMan—based on the gated fusion unit and Manhattan attention. GatedMan utilizes RoBERTa as a pre-trained model and enhances it using bidirectional long short-term memory (BiLSTM) to extract features from the context. It uses a gated unit to perform weighted fusion between the outputs of RoBERTa and BiLSTM, thereby enriching the information flow. The fused output is then fed into a novel Manhattan attention mechanism to capture the long-range dependencies. The global optimum tagging sequence is obtained using the conditional random fields layer. To enhance the model’s robustness, we incorporate adversarial training using the fast gradient method. This introduces adversarial examples, allowing the model to learn more disturbance-resistant feature representations, thereby improving its performance against unknown inputs. GatedMan achieved F1 scores of 93.73%, 94.13%, 93.98%, and 96.52% on the AgCNER, Peoples_daily, MSRA, and Resume datasets, respectively, thereby outperforming the other models. Experimental results demonstrate that GatedMan accurately identifies entities related to crop diseases and pests and exhibits high generalizability in other domains.

Application of machine reading comprehension techniques for named entity recognition in materials science

Materials science is an interdisciplinary field that studies the properties, structures, and behaviors of different materials. A large amount of scientific literature contains rich knowledge in the field of materials science, but manually analyzing these papers to find material-related data is a daunting task. In information processing, named entity recognition (NER) plays a crucial role as it can automatically extract entities in the field of materials science, which have significant value in tasks such as building knowledge graphs. The typically used sequence labeling methods for traditional named entity recognition in material science (MatNER) tasks often fail to fully utilize the semantic information in the dataset and cannot effectively extract nested entities. Herein, we proposed to convert the sequence labeling task into a machine reading comprehension (MRC) task. MRC method effectively can solve the challenge of extracting multiple overlapping entities by transforming it into the form of answering multiple independent questions. Moreover, the MRC framework allows for a more comprehensive understanding of the contextual information and semantic relationships within materials science literature, by integrating prior knowledge from queries. State-of-the-art (SOTA) performance was achieved on the Matscholar, BC4CHEMD, NLMChem, SOFC, and SOFC-Slot datasets, with F1-scores of 89.64%, 94.30%, 85.89%, 85.95%, and 71.73%, respectively in MRC approach. By effectively utilizing semantic information and extracting nested entities, this approach holds great significance for knowledge extraction and data analysis in the field of materials science, and thus accelerating the development of material science.Scientific contributionWe have developed an innovative NER method that enhances the efficiency and accuracy of automatic entity extraction in the field of materials science by transforming the sequence labeling task into a MRC task, this approach provides robust support for constructing knowledge graphs and other data analysis tasks.

Comparative Review of Tasks, Approaches and Tools for Automated Knowledge Extraction from the Texts of Scientific Publications

The purpose of this work is to review the existing technologies for automated knowledge extraction from scientific publications. The main tasks include an analysis of existing methods for automated knowledge extraction, as well as an overview of various software tools used to solve this problem. The article presents a description of the main approaches to automated knowledge extraction, such as machine learning, natural language processing and the development of various methodologies for building knowledge graphs. An analysis of existing sources showed that the main problems associated with automated knowledge extraction are the need to create a large amount of labeled data, the processing of complex structured data, and the need to develop new algorithms for working with such data.

A novel method to build knowledge graph models for the configuration and operation design of smart and connected industrial products

Abstract Smart and connected industrial products (SCIPs), characterized by their capabilities of self-monitoring, environment awareness, machine–machine/machine–human communication and collaboration, intelligent decision-making, etc., have become the fundamental elements for cyber-physical systems, digital twin, industrial internet of things, etc. Configuring the components in SCIPs and modeling their interaction and operation mechanisms are important during SCIPs design. However, existing product design methods were originally developed for none smart and connected products. This could limit the accuracy of SCIP modeling during the design stage and consequently, it may cause more reworks during the implementation stage of the designed SCIPs. In this regard, a SCIP configuration and operation design method is established, including (i) meta knowledge graph (KG)-based configuration of the components in the physical system and status monitoring system of a required SCIP, (ii) event-state swimlane flowchart-based analysis of the dynamic interaction, operation, and data monitoring mechanisms among the components, and (iii) event-state KG based modeling of the overall workflow, monitoring data self-updating and intelligent operation mechanisms of the SCIP. Compared with existing SCIP design methods, the work provides a specific method for not only the configuration of the static components in customized SCIPs, but also the dynamic interaction, data acquisition/storing/transmitting, and intelligent function implementation mechanisms of the configured SCIP using a kind of event-state KG. The event-state KG is both human-readable and computer-programmable, and it can self-update according to predefined reasoning algorithms during the operation of the SCIP. The configuration and operation design modeling of a robot-based grinding processing line is used as a case study.

Building knowledge graphs from technical documents using named entity recognition and edge weight updating neural network with triplet loss for entity normalization

Attempts to express information from various documents in graph form are rapidly increasing. The speed and volume in which these documents are being generated call for an automated process, based on machine learning techniques, for cost-effective and timely analysis. Past studies responded to such needs by building knowledge graphs or technology trees from the bibliographic information of documents, or by relying on text mining techniques in order to extract keywords and/or phrases. While these approaches provide an intuitive glance into the technological hotspots or the key features of the select field, there still is room for improvement, especially in terms of recognizing the same entities appearing in different forms so as to interconnect closely related technological concepts properly. In this paper, we propose to build a patent knowledge network using the United States Patent and Trademark Office (USPTO) patent filings for the semiconductor device sector by fine-tuning Huggingface’s named entity recognition (NER) model with our novel edge weight updating neural network. For the named entity normalization, we employ edge weight updating neural network with positive and negative candidates that are chosen by substring matching techniques. Experiment results show that our proposed approach performs very competitively against the conventional keyword extraction models frequently employed in patent analysis, especially for the named entity normalization (NEN) and document retrieval tasks. By grouping entities with named entity normalization model, the resulting knowledge graph achieves higher scores in retrieval tasks. We also show that our model is robust to the out-of-vocabulary problem by employing the fine-tuned BERT NER model.

Helio: A framework for implementing the life cycle of knowledge graphs

Building and publishing knowledge graphs (KG) as Linked Data, either on the Web or in private companies, has become a relevant and crucial process in many domains. This process requires that users perform a wide number of tasks conforming to the life cycle of a KG, and these tasks usually involve different unrelated research topics, such as RDF materialisation or link discovery. There is already a large corpus of tools and methods designed to perform these tasks; however, the lack of one tool that gathers them all leads practitioners to develop ad-hoc pipelines that are not generic and, thus, non-reusable. As a result, building and publishing a KG is becoming a complex and resource-consuming process. In this paper, a generic framework called Helio is presented. The framework aims to cover a set of requirements elicited from the KG life cycle and provide a tool capable of performing the different tasks required to build and publish KGs. As a result, Helio aims at providing users with the means for reducing the effort required to perform this process and, also, Helio aims to prevent the development of ad-hoc pipelines. Furthermore, the Helio framework has been applied in many different contexts, from European projects to research work.

Algorithms and methods for automated construction of knowledge graphs based on text sources

In this article, we present our path towards building knowledge graphs automatically from Russian texts. We explore various methodologies and libraries to extract triples, which are the fundamental building blocks of knowledge graphs. Our approach involves the use of libraries for analyzing morphological characteristics of words, such as PyMorphy and Yandex Mystem, to construct triples. We also utilize the NLP library spaCy to analyze text and build triples based on semantic relationships recognized by the library. However, we found that in some cases, we could not extract relationships from the text, leading us to use word2vec to define relationships. Unfortunately, the results obtained from word2vec were unsatisfactory and could not be used as relationships. We also encountered the problem of building triples from text due to the use of pronouns. To address this issue, we explored the use of coreference resolution libraries, but unfortunately, there are no working libraries available for the Russian language at this time. Our results highlight both positive and negative outcomes of applying these methodologies and libraries, providing insights into the challenges and opportunities of building knowledge graphs automatically from Russian texts.

Automatic knowledge extraction from Chinese electronic medical records and rheumatoid arthritis knowledge graph construction.

Knowledge graphs are a powerful tool for organizing knowledge, processing information and integrating scattered information, effectively visualizing the relationships among entities and supporting further intelligent applications. One of the critical tasks in building knowledge graphs is knowledge extraction. The existing knowledge extraction models in the Chinese medical domain usually require high-quality and large-scale manually labeled corpora for model training. In this study, we investigate rheumatoid arthritis (RA)-related Chinese electronic medical records (CEMRs) and address the automatic knowledge extraction task with a small number of annotated samples from CEMRs, from which an authoritative RA knowledge graph is constructed. After constructing the domain ontology of RA and completing manual labeling, we propose the MC-bidirectional encoder representation from transformers-bidirectional long short-term memory-conditional random field (BERT-BiLSTM-CRF) model for the named entity recognition (NER) task and the MC-BERT + feedforward neural network (FFNN) model for the entity extraction task. The pretrained language model (MC-BERT) is trained with many unlabeled medical data and fine-tuned using other medical domain datasets. We apply the established model to automatically label the remaining CEMRs, and then an RA knowledge graph is constructed based on the entities and entity relations, a preliminary assessment is conducted, and an intelligent application is presented. The proposed model achieved better performance than that of other widely used models in knowledge extraction tasks, with mean F1 scores of 92.96% in entity recognition and 95.29% in relation extraction. This study preliminarily confirmed that using a pretrained medical language model could solve the problem that knowledge extraction from CEMRs requires a large number of manual annotations. An RA knowledge graph based on the above identified entities and extracted relations from 1,986 CEMRs was constructed. Experts verified the effectiveness of the constructed RA knowledge graph. In this paper, an RA knowledge graph based on CEMRs was established, the processes of data annotation, automatic knowledge extraction, and knowledge graph construction were described, and a preliminary assessment and an application were presented. The study demonstrated the viability of a pretrained language model combined with a deep neural network for knowledge extraction tasks from CEMRs based on a small number of manually annotated samples.

A Novel Approach of Voterank-Based Knowledge Graph for Improvement of Multi-Attributes Influence Nodes on Social Networks

Abstract Recently, measuring users and community influences on social media networks play significant roles in science and engineering. To address the problems, many researchers have investigated measuring users with these influences by dealing with huge data sets. However, it is hard to enhance the performances of these studies with multiple attributes together with these influences on social networks. This paper has presented a novel model for measuring users with these influences on a social network. In this model, the suggested algorithm combines Knowledge Graph and the learning techniques based on the vote rank mechanism to reflect user interaction activities on the social network. To validate the proposed method, the proposed method has been tested through homogeneous graph with the building knowledge graph based on user interactions together with influences in real-time. Experimental results of the proposed model using six open public data show that the proposed algorithm is an effectiveness in identifying influential nodes.

User identification for knowledge graph construction across multiple online social networks

User identification across multiple online social networks is beneficial for building knowledge graphs. Under privacy protection considerations, researchers have shown increasing interest in user identification based on username similarity. However, existing solutions rely on manual features extracted by domain experts and do not exploit the deep semantic features of usernames. Moreover, existing solutions are limited to monolingual user names such as English or Chinese, ignoring other multilingual usernames. This paper proposes a multilingual pre-trained model-based username similarity method for user identification across multiple online social networks. First, we use many multilingual corpora to enable the model to learn more semantic information and extract deep semantic features of usernames. Then, fine-tuning is performed on our constructed dataset of multilingual usernames across multiple online social networks. Ultimately assess the similarity of user identities across multiple online social networks. Our method facilitates user identification with limited data. Finally, the efficiency of our model is verified on three constructed real-world multilingual username datasets across multiple online social networks and compared with existing state-of-the-art methods. Experimental results show that the proposed algorithm outperforms the compared algorithms.

THT: Transfer Head and Tail for Joint Entity Relation Extraction

Extracting entities and relations from unstructured text is the basic task of building knowledge graphs. Various methods based on sequence labeling have been proposed recently, but most of them use uniform token representations to identify entities. It is clear that the first and last words of the head and tail entities should have different representations. For this reason, we propose a method based on word interaction that provides different representations for the first and last words of entities. Specifically, initial token representation is obtained by BERT. Each token interacts with the next k tokens for the entity’s first token recognition, interacts with the previous k tokens for the entity’s last token recognition, and merges them to obtain the head entity. Then recognize the tail entities based on the head entities and form entity pairs. Finally, get the triples based on a simple multi-classification for relation determination. Extensive experimental results on four datasets show that THT performs better than state-of-the-art baselines.

ERGM: A multi-stage joint entity and relation extraction with global entity match

Joint entity and relation extraction is a fundamental and important task in the process of building knowledge graphs. At present, many researchers have proposed their own methods to solve this task, but these studies often have some limitations, such as irrelevant relation prediction, and lack of information interaction between the relation and entity. The complex model structure leads to inefficiency and does not make good use of the associations between the various subtasks. We propose a novel lightweight joint extraction model based on a global entity matching strategy. Specifically, the proposed framework contains three components: Relation Extraction Module, Relation Attention Based Entity Recognition Module and Global Entity Pairing Module. The Relation Extraction Module extracts candidate relations in the sentence, and the Relation Attention Based Entity Recognition Module introduces a relation attention mechanism based on the obtained candidate relations to fuse the information of the relations so as to better identify entities in the sentence. Then use entity vector representations to construct an affine transformation-based global entity matching matrix under a specific relation for triple extraction. Our model decomposes entity and relation extraction into three sub-tasks, which greatly simplifies the model structure, and the tasks are interrelated, making full use of the relevant information. In addition, we introduce a negative sampling strategy to alleviate the exposure bias problem of the model. We validate Our model on public dataset, it not only can effectively solve the triple overlap problem but also achieved a significant time performance speedup and effectively reduce memory occupation.

Research and Implementation of Automatic Indexing Method of PDF for Digital Publishing

With the rapid development of mobile Internet technology and artificial intelligence technology, the digital publishing industry is in urgent need of using intelligent technology to change the current way of content production and service. Most of the e-book resources owned by publishing enterprises are in PDF format, which is not suitable for reading on mobile devices, and it is not convenient to directly extract key information and construct knowledge graph. With this in mind, this article designs a PDF automatic indexing scheme that can identify all the element information in PDF and output structured data automatically and then extract all the key information in it to generate a keyword library with tag weights. The scheme mainly involves two key technical points: parsing PDF based on text features and grammar rules and extracting keywords based on tag weights. The former visualizes the text block in PDF into a rectangular area, divides the elements by clustering algorithm, and, finally, outputs structured data containing all the information. The latter combines the tags and their weights in the structured data and extracts the keywords in it by the inter-word relation algorithm. The structured data and keywords database produced by this scheme can be used to produce intelligent e-book and build knowledge graph, thus helping publishing enterprises to transform from a content service provider to an intelligent knowledge service provider. This transformation can deeply excavate the core value of the content held by the publishing industry and promote the digitization and intelligentization process of the whole industry.

Edge-Labeled and Node-Aggregated Graph Neural Networks for Few-Shot Relation Classification

Relation classification as a core technique for building knowledge graphs becomes a critical task in natural language processing. The fact that humans can learn by summarizing and generalizing limited knowledge motivates scholars to explore few-shot learning. Graph neural networks provide a method to measure the distance between nodes, which improves the model effect in the problem of few-shot relation classification. However, graph neural network methods focus only on node information and ignore edge information which implies inter-class and intra-class relations. This paper proposes edge-labeled and node-aggregated graph neural networks (ENGNNs) for few-shot relation classification: edge labels are encoded and used for node information aggregation. In addition, a process of semi-supervised learning is designed to discover a better solution for one-shot learning. Compared with previous methods, experimental results show that the proposed ENGNN model improves the performance of the graph neural network on the FewRel dataset.

Named entity recognition for Chinese based on global pointer and adversarial training

Named entity recognition aims to identify entities from unstructured text and is an important subtask for natural language processing and building knowledge graphs. Most of the existing entity recognition methods use conditional random fields as label decoders or use pointer networks for entity recognition. However, when the number of tags is large, the computational cost of method based on conditional random fields is high and the problem of nested entities cannot be solved. The pointer network uses two modules to identify the first and the last of the entities separately, and a single module can only focus on the information of the first or the last of the entities, but cannot pay attention to the global information of the entities. In addition, the neural network model has the problem of local instability. To solve mentioned problems, a named entity recognition model based on global pointer and adversarial training is proposed. To obtain global entity information, global pointer is used to decode entity information, and rotary relative position information is considered in the model designing to improve the model’s perception of position; to solve the model’s local instability problem, adversarial training is used to improve the robustness and generalization of the model. The experimental results show that the F1 score of the model are improved on several public datasets of OntoNotes5, MSRA, Resume, and Weibo compared with the existing mainstream models.

A graph neural network-enhanced knowledge graph framework for intelligent analysis of policing cases.

In this paper, we model a knowledge graph based on graph neural networks, conduct an in-depth study on building knowledge graph embeddings for policing cases, and design a graph neural network-enhanced knowledge graph framework. In detail, we use the label propagation algorithm (LPA) to assist the convolutional graph network (GCN) in training the edge weights of the knowledge graph to construct a policing case prediction method. This improves the traditional convolutional neural network from a single-channel network to a multichannel network to accommodate the multiple feature factors of policing cases. In addition, this expands the perceptual field of the convolutional neural network to improve prediction accuracy. The experimental results show that the multichannel convolutional neural network's prediction accuracy can reach 87.7%. To ensure the efficiency of the security case analysis network, an efficient pairwise feature extraction base module is added to enhance the backbone network, which reduces the number of parameters of the whole network and decreases the complexity of operations. We experimentally demonstrate that this method achieves a better balance of efficiency and performance by obtaining approximate results with 53.5% fewer floating-point operations and 70.2% fewer number parameters than its contemporary work.

Automated dynamic schema generation using knowledge graph

<span>On the internet where the number of database developers is increasing with the availability of huge data to be stored and queried. Establishing relations between various schemas and helping the developers by filtering, prioritizing, and suggesting relevant schema is a requirement. Recommendation system plays an important role in searching through a large volume of dynamically generated schemas to provide database developers with personalized schemas and services. Although many methods are already available to solve problems using machine learning, they require more time and data to learn. These problems can be solved using knowledge graphs (KG). This paper investigates building knowledge graphs to recommend schemas. </span>

Building Knowledge Graphs from Unstructured Texts: Applications and Impact Analyses in Cybersecurity Education

Knowledge graphs gained popularity in recent years and have been useful for concept visualization and contextual information retrieval in various applications. However, constructing a knowledge graph by scraping long and complex unstructured texts for a new domain in the absence of a well-defined ontology or an existing labeled entity-relation dataset is difficult. Domains such as cybersecurity education can harness knowledge graphs to create a student-focused interactive and learning environment to teach cybersecurity. Learning cybersecurity involves gaining the knowledge of different attack and defense techniques, system setup and solving multi-facet complex real-world challenges that demand adaptive learning strategies and cognitive engagement. However, there are no standard datasets for the cybersecurity education domain. In this research work, we present a bottom-up approach to curate entity-relation pairs and construct knowledge graphs and question-answering models for cybersecurity education. To evaluate the impact of our new learning paradigm, we conducted surveys and interviews with students after each project to find the usefulness of bot and the knowledge graphs. Our results show that students found these tools informative for learning the core concepts and they used knowledge graphs as a visual reference to cross check the progress that helped them complete the project tasks.

Building Knowledge Graphs and Recommender Systems for Suggesting Reskilling and Upskilling Options from the Web

As advances in science and technology, crisis, and increased competition impact labor markets, reskilling and upskilling programs emerged to mitigate their effects. Since information on continuing education is highly distributed across websites, choosing career paths and suitable upskilling options is currently considered a challenging and cumbersome task. This article, therefore, introduces a method for building a comprehensive knowledge graph from the education providers’ Web pages. We collect educational programs from 488 providers and leverage entity recognition and entity linking methods in conjunction with contextualization to extract knowledge on entities such as prerequisites, skills, learning objectives, and course content. Slot filling then integrates these entities into an extensive knowledge graph that contains close to 74,000 nodes and over 734,000 edges. A recommender system leverages the created graph, and background knowledge on occupations to provide a career path and upskilling suggestions. Finally, we evaluate the knowledge extraction approach on the CareerCoach 2022 gold standard and draw upon domain experts for judging the career paths and upskilling suggestions provided by the recommender system.

Deciphering the Diversity of Mental Models in Neurodevelopmental Disorders: Knowledge Graph Representation of Public Data Using Natural Language Processing.

BackgroundUnderstanding how individuals think about a topic, known as the mental model, can significantly improve communication, especially in the medical domain where emotions and implications are high. Neurodevelopmental disorders (NDDs) represent a group of diagnoses, affecting up to 18% of the global population, involving differences in the development of cognitive or social functions. In this study, we focus on 2 NDDs, attention deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD), which involve multiple symptoms and interventions requiring interactions between 2 important stakeholders: parents and health professionals. There is a gap in our understanding of differences between mental models for each stakeholder, making communication between stakeholders more difficult than it could be.ObjectiveWe aim to build knowledge graphs (KGs) from web-based information relevant to each stakeholder as proxies of mental models. These KGs will accelerate the identification of shared and divergent concerns between stakeholders. The developed KGs can help improve knowledge mobilization, communication, and care for individuals with ADHD and ASD.MethodsWe created 2 data sets by collecting the posts from web-based forums and PubMed abstracts related to ADHD and ASD. We utilized the Unified Medical Language System (UMLS) to detect biomedical concepts and applied Positive Pointwise Mutual Information followed by truncated Singular Value Decomposition to obtain corpus-based concept embeddings for each data set. Each data set is represented as a KG using a property graph model. Semantic relatedness between concepts is calculated to rank the relation strength of concepts and stored in the KG as relation weights. UMLS disorder-relevant semantic types are used to provide additional categorical information about each concept’s domain.ResultsThe developed KGs contain concepts from both data sets, with node sizes representing the co-occurrence frequency of concepts and edge sizes representing relevance between concepts. ADHD- and ASD-related concepts from different semantic types shows diverse areas of concerns and complex needs of the conditions. KG identifies converging and diverging concepts between health professionals literature (PubMed) and parental concerns (web-based forums), which may correspond to the differences between mental models for each stakeholder.ConclusionsWe show for the first time that generating KGs from web-based data can capture the complex needs of families dealing with ADHD or ASD. Moreover, we showed points of convergence between families and health professionals’ KGs. Natural language processing–based KG provides access to a large sample size, which is often a limiting factor for traditional in-person mental model mapping. Our work offers a high throughput access to mental model maps, which could be used for further in-person validation, knowledge mobilization projects, and basis for communication about potential blind spots from stakeholders in interactions about NDDs. Future research will be needed to identify how concepts could interact together differently for each stakeholder.