Fact Triples Research Articles

Multi-level feature interaction for open knowledge base canonicalization

Open Information Extraction (OpenIE) aims to construct expansive open knowledge bases (OKBs) by extracting triples (noun phrase, relation phrase, noun phrase) from unstructured text. One critical problem in OKBs is the lack of canonicalization for noun phrases and relation phrases, leading to the storage of redundant and ambiguous facts. Consequently, open knowledge base canonicalization, which clusters synonymous phrases into the same group, has emerged as an active research area. Existing approaches either leverage fact triples or source context in isolation, or at best, interact them at the clustering level. However, these approaches lack explicit interaction or only loosely couple the two types of knowledge, resulting in the potential loss of valuable intermediate information. In this paper, we propose MuFIC, a novel unsupervised framework that interacts the fact triples and source context at the feature level to address these limitations. In order to capture and integrate fine-grained fact and context knowledge, we design three levels of feature interaction: low-level context-guided feature interaction, mid-level fact-guided feature interaction, and high-level gated fusion feature interaction. Furthermore, we introduce an additional objective function via contrastive learning to improve the quality of extracted features and reduce knowledge-specific noise. Finally, we design a bidirectional feedback mechanism to better guide the learning process of joint features by harnessing side information prototype learning, and to dynamically optimize side information based on the clustering results formed by joint features. Extensive experiments on three public benchmark datasets demonstrate the superiority of our proposed framework.

KGRLFF: Detecting Drug-Drug Interactions Based on Knowledge Graph Representation Learning and Feature Fusion.

Accurate prediction of drug-drug interactions (DDIs) plays an important role in improving the efficiency of drug development and ensuring the safety of combination therapy. Most existing models rely on a single source of information to predict DDIs, and few models can perform tasks on biomedical knowledge graphs. This paper proposes a new hybrid method, namely Knowledge Graph Representation Learning and Feature Fusion (KGRLFF), to fully exploit the information from the biomedical knowledge graph and molecular structure of drugs to better predict DDIs. KGRLFF first uses a Bidirectional Random Walk sampling method based on the PageRank algorithm (BRWP) to obtain higher-order neighborhood information of drugs in the knowledge graph, including neighboring nodes, semantic relations, and higher-order information associated with triple facts. Then, an embedded representation learning model named Knowledge Graph-based Cyclic Recursive Aggregation (KGCRA) is used to learn the embedded representations of drugs by recursively propagating and aggregating messages with drugs as both the source and destination. In addition, the model learns the molecular structures of the drugs to obtain the structured features. Finally, a Feature Representation Fusion Strategy (FRFS) was developed to integrate embedded representations and structured feature representations. Experimental results showed that KGRLFF is feasible for predicting potential DDIs.

Distant Supervision-based Relation Extraction for Literature-Related Biomedical Knowledge Graph Construction

Background: The task of relation extraction is a crucial component in the construction of a knowledge graph. However, it often necessitates a significant amount of manual annotation, which can be time-consuming and expensive. Distant supervision, as a technique, seeks to mitigate this challenge by generating a large volume of pseudo-training data at a minimal cost, achieved by mapping triple facts onto the raw text. Objective: The aim of this study is to explore the novelty and potential of the distant supervisionbased relation extraction approach. By leveraging this innovative method, we aim to enhance knowledge reliability and facilitate new knowledge discovery, establishing associations between knowledge from specific biomedical data or existing knowledge graphs and literature. Methods: This study presents a methodology to construct a biomedical knowledge graph employing distant supervision techniques. Through establishing links between knowledge entities and relevant literature sources, we methodically extract and integrate information, thereby expanding and enriching the knowledge graph. This study identified five types of biomedical entities (e.g., diseases, symptoms and genes) and four kinds of relationships. These were linked to PubMed literature and divided into training and testing datasets. To mitigate data noise, the training set underwent preprocessing, while the testing set was manually curated. Results: In our research, we successfully associated 230,698 triples from the existing knowledge graph with relevant literature. Furthermore, we identified additional 205,148 new triples directly sourced from these studies. Conclusion: Our study markedly advances the field of biomedical knowledge graph enrichment, particularly in the context of Traditional Chinese Medicine (TCM). By validating a substantial number of triples through literature associations and uncovering over 200,000 new triples, we have made a significant stride in promoting the development of evidence-based medicine in TCM. The results underscore the potential of using a distant supervision-based relation extraction approach to both validate and expand knowledge bases, contributing to the broader progression of evidence-based practices in the realm of TCM.

Improving Extraction of Chinese Open Relations Using Pre-trained Language Model and Knowledge Enhancement

ABSTRACT Open Relation Extraction (ORE) is a task of extracting semantic relations from a text document. Current ORE systems have significantly improved their efficiency in obtaining Chinese relations, when compared with conventional systems which heavily depend on feature engineering or syntactic parsing. However, the ORE systems do not use robust neural networks such as pre-trained language models to take advantage of large-scale unstructured data effectively. In respons to this issue, a new system entitled Chinese Open Relation Extraction with Knowledge Enhancement (CORE-KE) is presented in this paper. The CORE-KE system employs a pre-trained language model (with the support of a Bidirectional Long Short-Term Memory (BiLSTM) layer and a Masked Conditional Random Field (Masked CRF) layer) on unstructured data in order to improve Chinese open relation extraction. Entity descriptions in Wikidata and additional knowledge (in terms of triple facts) extracted from Chinese ORE datasets are used to fine-tune the pre-trained language model. In addition, syntactic features are further adopted in the training stage of the CORE-KE system for knowledge enhancement. Experimental results of the CORE-KE system on two large-scale datasets of open Chinese entities and relations demonstrate that the CORE-KE system is superior to other ORE systems. The F1-scores of the CORE-KE system on the two datasets have given a relative improvement of 20.1% and 1.3%, when compared with benchmark ORE systems, respectively. The source code is available at https://github.com/cjwen15/CORE-KE.

A confidence-aware and path-enhanced convolutional neural network embedding framework on noisy knowledge graph

In recent years, Knowledge Graphs (KGs) have been widely used in applications such as search engines and Q&A systems. During construction, errors were inevitably introduced and KGs may contain many incorrect facts. However, most KG representation learning methods assume that all triple facts are correct, making KG noise detection a challenge. Although some knowledge representation methods are based on a confidence-aware framework, they are usually limited by the translation assumption in TransE and confidence calculation methods. To address the above problems, we propose a more accurate and effective confidence-aware and path-enhanced convolutional neural network knowledge embedding framework (CPConvKE). It makes use of structural, entity type, and rule information for knowledge graph noise detection and knowledge representation learning simultaneously. Specifically, for representation learning, we introduce a gating-based path embedding method to filter out noise while learning entity and relation embeddings. For the confidence evaluation, we propose a triple confidence estimator that uses entity type and rule information as prior probability and defines posterior probability based on the embedding. We evaluate the effectiveness of our model on the knowledge graph noise detection, completion, and triple classification tasks. The experimental results show that our confidence-aware model achieves significant and consistent improvements on all tasks, which confirms the ability of CPConvKE to detect noise and learn clean embeddings in a noisy knowledge graph.

Generic User Behavior: A User Behavior Similarity-Based Recommendation Method

Recommender system (RS) plays an important role in Big Data research. Its main idea is to handle huge amounts of data to accurately recommend items to users. The recommendation method is the core research content of the whole RS. However, the existing recommendation methods still have the following two shortcomings: (1) Most recommendation methods use only one kind of information about the user's interaction with items (such as Browse or Purchase), which makes it difficult to model complete user preference. (2) Most mainstream recommendation methods only consider the final consistency of recommendation (e.g., user preferences) but ignore the process consistency (e.g., user behavior), which leads to the biased final result. In this article, we propose a recommendation method based on the Entity Interaction Knowledge Graph (EIKG), which draws on the idea of collaborative filtering and innovatively uses the similarity of user behaviors to recommend items. The method first extracts fact triples containing interaction relations from relevant data sets to generate the EIKG; then embeds the entities and relations in the EIKG; finally, uses link prediction techniques to recommend items for users. The proposed method is compared with other recommendation methods on two publicly available data sets, Scholat and Lizhi, and the experimental result shows that it exceeds the state of the art in most metrics, verifying the effectiveness of the proposed method.

Knowledge graph embedding learning system for defect diagnosis in additive manufacturing

Since additive manufacturing involves complex physical processes and is influenced by many process parameters, it is challenging to manufacture defect-free products. To explore the causes of defects and obtain better product quality, exploring the relations among these factors is necessary. The knowledge graph is a powerful technique to express the relationships between entities and has been applied to many real-world tasks, such as disease diagnosis, recommendation systems, and traffic prediction. In this paper, a novel knowledge graph embedding learning system is presented, in which a designed knowledge graph ontology for AM and fact triples collected from the literature are utilized for the construction of a knowledge graph for defect diagnosis in additive manufacturing (KDDAM). A relational enhanced graph convolutional network is proposed to uncover new hidden relations among parameters, effects, defects, and product quality in the KDDAM. To validate the performance of the proposed model, we conducted many experiments on KDDAM and compared it with other baseline and state-of-the-art models. Our model yielded better results in two traditional link prediction metrics, with an MRR of 0.316 and 0.7, respectively. In addition, a case study shows that our model can discover new causes of defects to help us optimize the process.

Towards Robust Knowledge Graph Embedding via Multi-Task Reinforcement Learning

Nowadays, Knowledge graphs (KGs) have been playing a pivotal role in AI-related applications. Despite the large sizes, existing KGs are far from complete and comprehensive. In order to continuously enrich KGs, automatic knowledge construction and update mechanisms are usually utilized, which inevitably bring in plenty of noise. However, most existing knowledge graph embedding (KGE) methods assume that all the triple facts in KGs are correct, and project both entities and relations into a low-dimensional space without considering noise and knowledge conflicts. This will lead to low-quality and unreliable representations of KGs. To this end, in this paper, we propose a general multi-task reinforcement learning framework, which can greatly alleviate the noisy data problem. In our framework, we exploit reinforcement learning for choosing high-quality knowledge triples while filtering out the noisy ones. Also, in order to take full advantage of the correlations among semantically similar relations, the triple selection processes of similar relations are trained in a collective way with multi-task learning. Moreover, we extend popular KGE models TransE, DistMult, ConvE and RotatE with the proposed framework. Finally, the experimental validation shows that our approach is able to enhance existing KGE models and can provide more robust representations of KGs in noisy scenarios.

A lightweight CNN-based knowledge graph embedding model with channel attention for link prediction.

Knowledge graph (KG) embedding is to embed the entities and relations of a KG into a low-dimensional continuous vector space while preserving the intrinsic semantic associations between entities and relations. One of the most important applications of knowledge graph embedding (KGE) is link prediction (LP), which aims to predict the missing fact triples in the KG. A promising approach to improving the performance of KGE for the task of LP is to increase the feature interactions between entities and relations so as to express richer semantics between them. Convolutional neural networks (CNNs) have thus become one of the most popular KGE models due to their strong expression and generalization abilities. To further enhance favorable features from increased feature interactions, we propose a lightweight CNN-based KGE model called IntSE in this paper. Specifically, IntSE not only increases the feature interactions between the components of entity and relationship embeddings with more efficient CNN components but also incorporates the channel attention mechanism that can adaptively recalibrate channel-wise feature responses by modeling the interdependencies between channels to enhance the useful features while suppressing the useless ones for improving its performance for LP. The experimental results on public datasets confirm that IntSE is superior to state-of-the-art CNN-based KGE models for link prediction in KGs.

Convolutional Neural Network Knowledge Graph Link Prediction Model Based on Relational Memory.

A knowledge graph is a collection of fact triples, a semantic network composed of nodes and edges. Link prediction from knowledge graphs is used to reason about missing parts of triples. Common knowledge graph link prediction models include translation models, semantics matching models, and neural network models. However, the translation models and semantic matching models have relatively simple structures and poor expressiveness. The neural network model can easily ignore the overall structural characteristics of triples and cannot capture the links between entities and relations in low-dimensional space. In response to the above problems, we propose a knowledge graph embedding model based on a relational memory network and convolutional neural network (RMCNN). We encode triple embedding vectors using a relational memory network and decode using a convolutional neural network. First, we will obtain entity and relation vectors by encoding the latent dependencies between entities and relations and some critical information and keeping the translation properties of triples. Then, we compose a matrix of head entity encoding embedding vector, relation encoding embedding vector, and tail entity embedding encoding vector as the input of the convolutional neural network. Finally, we use a convolutional neural network as the decoder and a dimension conversion strategy to improve the information interaction capability of entities and relations in more dimensions. Experiments show that our model achieves significant progress and outperforms existing models and methods on several metrics.

Sequence Encoder-based Spatiotemporal Knowledge Graph Completion

Knowledge graph (KG) completion aims to infer new facts from incomplete knowledge graphs. Most existing solutions focus on learning from time-aware fact triples and ignore the spatial information. In reality, knowledge graphs can evolve with time as well as the changing locations, such as the flight domain. Therefore, integrating spatiotemporal information into knowledge graph representation is important for the knowledge graph completion. To address this problem, this paper proposes two Spatio Temporal-aware knowledge graph completion models based on the Sequence Encoder, namely STSE and S-TSE, which incorporate the spatial and temporal information into relations. Specifically, the model consists of two steps: spatiotemporal-aware relation encoding and final scoring function evaluation. The first stage composes the spatiotemporal information into different tokens. Then two methods are proposed to obtain the embedding of spatiotemporal-aware relation by utilizing the Recursive Neural Network. The second stage proposes different scoring functions for two models. Empirically evaluation of the proposed models is conducted on spatiotemporal-aware KG completion task on two public datasets. Experimental results demonstrate the effectiveness of the proposal for spatiotemporal knowledge graph completion.

Fusing topology contexts and logical rules in language models for knowledge graph completion

Knowledge graph completion (KGC) aims to infer missing facts based on the observed ones, which is significant for many downstream applications. Given the success of deep learning and pre-trained language models (LMs), some LM-based methods are proposed for the KGC task. However, most of them focus on modeling the text of fact triples and ignore the deeper semantic information (e.g., topology contexts and logical rules) that is significant for KG modeling. For such a reason, we propose a unified framework FTL-LM to Fuse Topology contexts and Logical rules in Language Models for KGC, which mainly contains a novel path-based method for topology contexts learning and a variational expectation–maximization (EM) algorithm for soft logical rule distilling. The former utilizes a heterogeneous random-walk to generate topology paths and further reasoning paths that can represent topology contexts implicitly and can be modeled by a LM explicitly. The strategies of mask language modeling and contrastive path learning are introduced to model these topology contexts. The latter implicitly fuses logical rules by a variational EM algorithm with two LMs. Specifically, in the E-step, the triple LM is updated under the supervision of observed triples and valid hidden triples verified by the fixed rule LM. And in the M-step, we fix the triple LM and fine-tune the rule LM to update logical rules. Experiments on three common KGC datasets demonstrate the superiority of the proposed FTL-LM, e.g., it achieves 2.1% and 3.1% Hits@10 improvement over the state-of-the-art LM-based model LP-BERT in the WN18RR and FB15k-237, respectively.

Attention-based Knowledge Graph Representation Learning for Predicting Drug-drug Interactions.

Drug-drug interactions (DDIs) are known as the main cause of life-threatening adverse events, and their identification is a key task in drug development. Existing computational algorithms mainly solve this problem by using advanced representation learning techniques. Though effective, few of them are capable of performing their tasks on biomedical knowledge graphs (KGs) that provide more detailed information about drug attributes and drug-related triple facts. In this work, an attention-based KG representation learning framework, namely DDKG, is proposed to fully utilize the information of KGs for improved performance of DDI prediction. In particular, DDKG first initializes the representations of drugs with their embeddings derived from drug attributes with an encoder-decoder layer, and then learns the representations of drugs by recursively propagating and aggregating first-order neighboring information along top-ranked network paths determined by neighboring node embeddings and triple facts. Last, DDKG estimates the probability of being interacting for pairwise drugs with their representations in an end-to-end manner. To evaluate the effectiveness of DDKG, extensive experiments have been conducted on two practical datasets with different sizes, and the results demonstrate that DDKG is superior to state-of-the-art algorithms on the DDI prediction task in terms of different evaluation metrics across all datasets.

Knowledge structure driven prototype learning and verification for fact checking

To inhibit the spread of rumorous information, fact checking aims at retrieving relevant evidence to verify the veracity of a given claim. Previous work on fact checking typically uses knowledge graphs (KGs) as external repositories and develop reasoning methods to retrieve evidence from KGs. Domain knowledge structure, including category hierarchy and attribute relationships, can be utilized as discriminative information to facilitate KG based learning and verification. However, in previous fact checking research, category hierarchy and attribute information was often scattered in a KG and treated as the ordinary triple facts in the learning process like other types of information, or was utilized in a limited way without the consideration of category hierarchy or the combination of category hierarchy with the learning process. Thus to better utilize category hierarchy and attribute relationships, in this paper, we propose an end-to-end knowledge structure driven prototype learning and verification method for fact checking. For improving intra-category compactness and inter-category separation, we develop a hierarchical prototype learning technique that jointly learns a prototype for each sub-category to enhance entity embeddings and optimize embedding representations using high-level category. For further enhancing embedding learning, we propose a graph attention network to aggregate information from neighboring attribute nodes. We construct a real-world dataset on food domain, and experimental results on the benchmark datasets and our domain dataset show the effectiveness of our method compared to both previous fact checking methods and representative KG reasoning methods.

Modeling Relation Paths for Knowledge Graph Completion

Knowledge graphs (KG) often encounter knowledge incompleteness. The path reasoning that predicts the unknown path relation between pairwise entities based on existing facts is one of the most promising approaches to the knowledge graph completion. However, most conventional path reasoning methods exclusively consider the entity description included in fact triples, ignoring both the type information of entities and the interaction between different semantic representations. In this study, we propose a novel method, Type-aware Attentive Path Reasoning (TAPR), to complete the knowledge graph by simultaneously considering KG structural information, textual information, and type information. More specifically, we first leverage types to enrich the representational learning of entities and relationships. Next, we describe a type-level attention to select the most relevant type of given entity in a specific triple without any predefined rules or patterns to reduce the impact of noisy types. After learning the distributed representation of all paths, path-level attention assigns different weights to paths, from which relations among entity pairs are calculated. We conduct a series of experiments on a real-world dataset to demonstrate the effectiveness of TAPR. Experimental results show that our method significantly outperforms all baselines on link prediction and entity prediction tasks.

QB4MobOLAP: A Vocabulary Extension for Mobility OLAP on the Semantic Web

The accessibility of devices that track the positions of moving objects has attracted many researchers in Mobility Online Analytical Processing (Mobility OLAP). Mobility OLAP makes use of trajectory data warehousing techniques, which typically include a path of moving objects at a particular point in time. The Semantic Web (SW) users have published a large number of moving object datasets that include spatial and non-spatial data. These data are available as open data and require advanced analysis to aid in decision making. However, current SW technologies support advanced analysis only for multidimensional data warehouses and Online Analytical Processing (OLAP) over static spatial and non-spatial SW data. The existing technology does not support the modeling of moving object facts, the creation of basic mobility analytical queries, or the definition of fundamental operators and functions for moving object types. This article introduces the QB4MobOLAP vocabulary, which enables the analysis of mobility data stored in RDF cubes. This article defines Mobility OLAP operators and SPARQL user-defined functions. As a result, QB4MobOLAP vocabulary and the Mobility OLAP operators are evaluated by applying them to a practical use case of transportation analysis involving 8826 triples consisting of approximately 7000 fact triples. Each triple contains nearly 1000 temporal data points (equivalent to 7 million records in conventional databases). The execution of six pertinent spatiotemporal analytics query samples results in a practical, simple model with expressive performance for the enabling of executive decisions on transportation analysis.

TransET: Knowledge Graph Embedding with Entity Types

Knowledge graph embedding aims to embed entities and relations into low-dimensional vector spaces. Most existing methods only focus on triple facts in knowledge graphs. In addition, models based on translation or distance measurement cannot fully represent complex relations. As well-constructed prior knowledge, entity types can be employed to learn the representations of entities and relations. In this paper, we propose a novel knowledge graph embedding model named TransET, which takes advantage of entity types to learn more semantic features. More specifically, circle convolution based on the embeddings of entity and entity types is utilized to map head entity and tail entity to type-specific representations, then translation-based score function is used to learn the presentation triples. We evaluated our model on real-world datasets with two benchmark tasks of link prediction and triple classification. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

A reasoning enhance network for muti-relation question answering

Multi-relation Question Answering is an important task of knowledge base over question answering (KBQA), multi-relation means that the question contains multiple relations and entity information, so it needs to use the fact triples in the knowledge base to analyze and reasoning the question in more detail. In this paper, we propose a novel model called Reasoning Enhance Network that uses context information, enhance the accuracy of relation and entity predicted in each hop. The model obtains the relation by analyzing the context information before each hop start, and then reasons the answer by the previous information; update question representation and reasoning state through predicted relation and entity, then promote the next hop reasoning starts. Our experiments clearly show that our method achieves good results on four datasets. Also, since we use attention mechanisms, our method offers better interpretability.

A Knowledge Graph Embedding Approach for Metaphor Processing

Metaphor is a figure of speech that describes one thing (a target) by mentioning another thing (a source) in a way that is not literally true. Metaphor understanding is an interesting but challenging problem in natural language processing. This paper presents a novel method for metaphor processing based on knowledge graph (KG) embedding. Conceptually, we abstract the structure of a metaphor as an attribute-dependent relation between the target and the source. Each specific metaphor can be represented as a metaphor triple (target, attribute, source). Therefore, we can model metaphor triples just like modeling fact triples in a KG and exploit KG embedding techniques to learn better representations of concepts, attributes and concept relations. In this way, metaphor interpretation and generation could be seen as KG completion, while metaphor detection could be viewed as a representation learning enhanced concept pair classification problem. Technically, we build a Chinese metaphor KG in the form of metaphor triples based on simile recognition, and also extract concept-attribute collocations to help describe concepts and measure concept relations. We extend the translation-based and the rotation-based KG embedding models to jointly optimize metaphor KG embedding and concept-attribute collocation embedding. Experimental results demonstrate the effectiveness of our method. Simile recognition is feasible for building the metaphor triple resource. The proposed models improve the performance on metaphor interpretation and generation, and the learned representations also benefit nominal metaphor detection compared with strong baselines.

Aggregating neighborhood information for negative sampling for knowledge graph embedding

Knowledge graphs, as linked data, can be extracted from texts in triple form that illustrate the structure of “entity–relation–entity.” Knowledge graph embedding (KGE) models are used to map entities and relations into a continuous vector space with semantic constraints so as to learn a knowledge graph with fact triples. In the KGE model training process, both positive and negative triples are necessarily provided. Thus, negative sampling methods are meaningful in generating negative samples based on the representations of entities and relations. This paper proposes an innovative neighborhood knowledge selective adversarial network (NKSGAN), which leverages the representation of aggregating neighborhood information to generate high-quality negative samples for enhancing the performances of the discriminator. Experiments are conducted on widely used standard datasets such as FB15k, FB15k-237, WN18 and WN18RR to evaluate our model for link prediction task. The results present the superiority of our proposed NKSGAN than other baseline methods, indicating that the negative sampling process in NKSGAN is effective in generating high-quality negative samples for boosting KGE models.