Knowledge Graph Representation Research Articles

Natural Language Processing (NLP) in Recommendation Systems

This paper delves into the application of Natural Language Processing (NLP) techniques in recommendation systems, specifically focusing on novel approaches to enhance recommendation accuracy and user satisfaction. The utilization of NLP algorithms has revolutionized how content is recommended to users, leveraging linguistic analysis and machine learning to understand user preferences and provide tailored suggestions. Our research explores various NLP methodologies, including sentiment analysis, topic modelling, and semantic analysis, to extract meaningful insights from textual data. Furthermore, we investigate the integration of deep learning models, such as neural networks and transformer architectures, to capture complex patterns and improve recommendation precision. A key highlight of our study is the introduction of a novel recommendation method termed "Knowledge Graph Embedding for Contextual Recommendation." This innovative approach combines knowledge graph representation with contextual understanding, allowing for more nuanced and personalized recommendations based on user interactions, historical data, and contextual relevance. We delve into the intricacies of this technique, detailing its implementation, training process, and evaluation metrics.

Knowledge graph representation learning with relation-guided aggregation and interaction

Knowledge graph representation learning (KGRL) aims to study the feature representations to solve the incompleteness of knowledge graphs (KGs). Recently, graph neural network (GNN) has achieved satisfactory performance in KGRL tasks. However, many GNN based KGRL approaches fail to capture the various semantic of entities, which may reduce the representation quality of embeddings. Furthermore, most of them treat relations equally when learning embeddings, which will lead to weakening of the specific semantic features. To alleviate the above problems, we propose a novel KGRL method (RGAI) with relation-guided aggregation (RGA) and relation-guided interaction (RGI). To capture rich attributes of entities, the RGA module generates multiple sub-structures according to relation types, and aggregates the semantic information among them. To mine specific semantic features and dig out the different contribution of relations, the RGI module calculates the importance of each relation to the central entity, and generates final entity features through aggregating representations of neighbors. Extensive experiments on standard KG datasets validate the effectiveness of the proposed innovations, and RGAI achieves state-of-the-art performance compared to existing approaches (e.g., compared to state-of-the-art approaches, RGAI improves Hits@10 from 0.548 to 0.562 on FB15K-237, Hits@3 from 0.561 to 0.582 on YAGO3-10, and Hits@1 from 0.449 to 0.455 on WN18RR).

An Improved Knowledge Graph Representation for Ransomware Attacks Using LSTM-Based Named Entity Relation Technique and Twitter Data

This research aims to examine the challenges associated with ransomware knowledge and develop a knowledge graph of ransomware attacks utilizing Twitter data. Ransomware is a constantly evolving global threat. The three main steps involved in the development of a knowledge graph from informal text are data collection and preprocessing, features extraction, and relation extraction. A ransomware ontology previously proposed has been used in this work for the extraction of the ransomware entities from unstructured data; this is aimed at making it fit the attacks reported on Twitter. The next process is the identification of the existing relationships in the dataset for the knowledge graph construction; the output of the process, which is the developed knowledge graph is evaluated for accuracy using a tracing technique to demonstrate its efficacy.

Evolving Knowledge Graph Representation Learning with Multiple Attention Strategies for Citation Recommendation System

The growing number of publications in the field of artificial intelligence highlights the need for researchers to enhance their efficiency in searching for relevant articles. Most paper recommendation models either rely on simplistic citation relationships among papers or focus on content-based approaches, both of which overlook interactions within academic networks. To address the aforementioned problem, knowledge graph embedding (KGE) methods have been used for citation recommendations because recent research proves that graph representations can effectively improve recommendation model accuracy. However, academic networks are dynamic, leading to changes in the representations of users and items over time. The majority of KGE-based citation recommendations are primarily designed for static graphs, thus failing to capture the evolution of dynamic knowledge graph (DKG) structures. To address these challenges, we introduced the evolving knowledge graph embedding (EKGE) method. In this methodology, evolving knowledge graphs are input into time-series models to learn the patterns of structural evolution. The model has the capability to generate embeddings for each entity at various time points, thereby overcoming limitation of static models that require retraining to acquire embeddings at each specific time point. To enhance the efficiency of feature extraction, we employed a multiple attention strategy. This helped the model find recommendation lists that are closely related to a user’s needs, leading to improved recommendation accuracy. Various experiments conducted on a citation recommendation dataset revealed that the EKGE model exhibits a 1.13% increase in prediction accuracy compared to other KGE methods. Moreover, the model’s accuracy can be further increased by an additional 0.84% through the incorporation of an attention mechanism.

Knowledge Graph Representation of Multi-Source Urban Storm Surge Hazard Information Based on Spatio-Temporal Coding and the Hazard Events Ontology Model

Coastal cities are increasingly vulnerable to urban storm surge hazards and the secondary hazards they cause (e.g., coastal flooding). Accurate representation of the spatio-temporal process of hazard event development is essential for effective emergency response. However, current knowledge graph representations face the challenge of integrating multi-source information with various spatial and temporal scales. To address this challenge, we propose a new information model for storm surge hazard events, involving a two-step process. First, a hazard event ontology is designed to model the components and hierarchical relationships of hazard event information. Second, we utilize multi-scale time segment integer coding and geographical coordinate subdividing grid coding to create a spatio-temporal framework, for modeling spatio-temporal features and spatio-temporal relationships. Using the 2018 typhoon Mangkhut storm surge event in Shenzhen as a case study and the hazard event information model as a schema layer, a storm surge event knowledge graph is constructed, demonstrating the integration and formal representation of heterogeneous hazard event information and enabling the fast retrieval of disasters in a given spatial or temporal range.

Power data quality assessment and verification governance based on knowledge graph

In addressing the challenges of scattered data and limited professional knowledge in traditional power data quality assessment and verification governance, our approach leveraged natural language processing (NLP) technology for text preprocessing, incorporating power system research findings and case analyses. Named entity recognition enhanced entity identification accuracy, while relationship weighting technology facilitated entity classification and relationship weight assignment. The resulting power system knowledge graph was seamlessly integrated into a graph database for real-time updates. Through the synergistic use of relationship weighting technology and graph convolutional networks (GCN), our method achieved precise representation and modeling of power system knowledge graphs. The research outcomes underscored the method’s exceptional performance in real-time anomaly detection, maintaining an anomaly detection rate between 1% and 10% and accuracy fluctuating from 80% to 98%. This method stands as a testament to its efficacy in processing power data and validating its robustness in power system assessment and verification.

MvTuckER: Multi-view knowledge graphs representation learning based on tensor tucker model

Recently, more and more multi-view knowledge graphs with various important attributes are being constructed and applied, such as temporal information, geolocation, commonsense knowledge, multi-language and multi-modal. Knowledge graphs (KGs) have been widely applied in question answering systems, drug discovery, information retrieval, and so on. Knowledge graph representation learning (KGRL) is emerging as the most effective approach to tasks such as knowledge graph completion, entity alignment, knowledge reasoning and knowledge query. However, most previous work on KGs and KGRL are mainly designed for traditional KGs and only consider single-view, and the KGRL methods for special multi-view KGs faces the challenges of lacking a unified framework, insufficient interactive information and underutilizing important attributes. To bridge this gap, in this paper, we proposes a multi-view knowledge graphs (e.g., temporal KGs) representation learning method based on tensor tucker model (MvTuckER), which models the multi-view KGs from tuples to a nth-order binary tensor, and adopts the tensor n-mode product to capture complete interaction between and within different views. We also introduce low-rank and sparse approximations of core tensors to balance expressivity and complexity of MvTuckER. Moreover, we theoretically illustrate that our model is fully expressive for modeling multi-view KGs, discuss the working mechanism of MvTuckER from the perspective of logical operation, and explain how to migrate and extend our method to other types of views. We conducted extensive experiments on three multi-view knowledge graphs, and obtain 4.7%, 2.7%, and 5.3% effective improvement in Hit@1 respectively, which proves that the proposed MvTuckER can achieve state-of-the-art performance.

Knowledge Graph Construction and Representation Method for Potato Diseases and Pests

Potato diseases and pests have a serious impact on the quality and yield of potatoes, and timely prevention and control of potato diseases and pests is essential. A rich knowledge reserve of potato diseases and pests is one of the most important prevention and control measures; however, valuable knowledge is buried in the massive data of potato diseases and pests, making it difficult for potato growers and managers to obtain and use it in a timely manner and to develop the potential of knowledge. Therefore, this paper explores the construction method of a knowledge graph for automatic knowledge extraction, which extracts the knowledge of potato diseases and pests scattered in heterogeneous data from multiple sources, organises it into a semantically related knowledge base, and provides potato growers with professional knowledge and timely guidance to effectively prevent and control potato diseases and pests. In this paper, a data corpus on potato diseases and pests, called PotatoRE, is first constructed. Then, a model of ALBert-BiLSTM-Self_Att-CRF is designed to extract knowledge from the corpus to form a triplet structure, which is imported into the Neo4j graph database for storage and visualisation. Furthermore, the performance of the model constructed in this paper is compared and verified using the datasets PotatoRE and People’s Daily. The results show that compared to the SOTA models of ALBert BiLSTM-CRF and ALBert BiGRU-CRF, the accuracy of our model has been improved by 2.92% and 3.12%, respectively, using PotatoRE. Compared to the Bert BiLSTM-CRF model on two datasets, our model not only improves the accuracy, recall, and F1 values, but also has a higher efficiency. The model in this paper solves the problem of the difficult recognition of nested entities. On this basis, through comparative experiments, the TransH model is used to effectively represent the constructed knowledge graph, which lays the foundation for achieving inference, extension, and automatic updating of the knowledge base. The achievements of the thesis have made certain contributions to the automatic construction of large-scale knowledge bases.

Knowledge graph representation method for semantic 3D modeling of Chinese grottoes

The integration of 3D geometric models with semantic information significantly improves the applicability and comprehensibility of cultural heritage. The semantic 3D modeling of Chinese grottoes poses challenges for individuals without expertise in cultural heritage due to gaps in domain knowledge and discrepancies in understanding. However, the existing domain ontology and knowledge graph provide an insufficient representation of the knowledge of Chinese grottoes. To overcome these obstacles, we propose a knowledge graph representation method to provide explicit knowledge for participants at different stages of semantic 3D modeling of Chinese grottoes, which includes schema layer construction and data layer construction. On the schema layer, we design a domain ontology named ChgOnto (Chinese Grottoes Ontology) that consists of four high-level concept classes: spatial object, informational object, digital device, and temporal object. Among the classes in the ChgOnto, the components (e.g., cliff wall, cave roof, cliff wall footing), elements (e.g., primary Buddha statue, pedestal, decoration), the properties (e.g., length, width, depth) of caves and niches in Chinese grottoes as well as the spatial relationships between them are all precisely defined. ChgOnto also reuse the classes from the renowned CIDOC CRM ontology in the cultural heritage field and GeoSPARQL in the geospatial domain, facilitating integration between the two subjects. Considering the schema layer as the conceptual data model, the data layer extracts knowledge from unstructured text through natural language processing tools to instantiate the abstract classes and fill the properties of the schema layer. Finally, the knowledge required for semantic 3D modeling of Chinese grottoes is expressed in the data layer by a knowledge graph in a fixed expression form. Dazu Rock Carvings, a World Heritage site in China, is selected as a case study to validate the practicality and effectiveness of the proposed method. The results reveal that our method offers a robust knowledge-sharing platform for the semantic 3D modeling of Chinese grottoes and demonstrates excellent scalability. The method proposed in this paper can also serve as an informative reference for other types of cultural heritage.

AugGKG: a grid-augmented geographic knowledge graph representation and spatio-temporal query model

ABSTRACTAs an emerging knowledge representation model in the domain of knowledge graphs, geographic knowledge graph can take full advantage of semantic, spatial and temporal information to facilitate answering spatio-temporal questions and completing relations. However, the representation of geographic knowledge graphs still has issues such as the difficulty of unified heterogeneous spatio-temporal data modelling, weak ability to answer spatio-temporal queries for dynamic multiobjective problems, and low efficiency of graph querying. This paper presents a grid-augmented geographic knowledge graph (AugGKG) based on the GeoSOT global subdivision grid model and time slice subgraph architecture. AugGKG discretely normalizes the spatio-temporal data of the graph, which involves five types of nodes and two types of relations. By using the geo-hidden layer of the graph and geocoding algebraic operations, the AugGKG can quickly answer complex multiobjective spatio-temporal queries and complete implicit spatio-temporal relations. Compared with existing geographic knowledge graphs (YAGO, GeoKG and GEKG), the comparative experiments verified the obvious advantages of AugGKG in terms of uniformity of accuracy, completeness, and efficiency. Hence, AugGKG is expected to be regarded as an innovative and robust geographic knowledge graph that can perform fast computation and relation completion for complex spatio-temporal queries in future geospatial question answering applications.

Knowledge Graph Representation in Medical Era-An Analysis

Healthcare knowledge graphs (HKGs) are considered as a tool for organizing medical knowledge in a structured and interpretable way, which provides a comprehensive view of medical concepts and their relationships. However, challenges such as data heterogeneity and limited coverage remain, emphasizing the need for further research in the field of HKGs.Here summarize the pipeline and key techniques for HKG construction, as well as the common utilization approaches (i.e., model free and model-based). To provide researchers with valuable resources, we organize existing HKGs1 based on the data types they capture and application domains, supplemented with pertinent statistical information. In the application section, we delve into the transformative impact of HKGs across various healthcare domains, spanning from fine-grained basic science research to high-level clinical decision support. Lastly, we shed light on the opportunities for creating comprehensive and accurate HKGs in the era of large language models, presenting the potential to revolutionize healthcare delivery and enhance the interpretability and reliability of clinical prediction

An inductive knowledge graph embedding via combination of subgraph and type information

Conventional knowledge graph representation learn the representation of entities and relations by projecting triples in the knowledge graph to a continuous vector space. The vector representation increases the precision of link prediction and the efficiency of downstream tasks. However, these methods cannot process previously unseen entities during the knowledge graph evolution. In other words, the model trained on the source knowledge graph cannot be applied to the target knowledge graph containing new unseen entities. Recently, a few subgraph-based link prediction models obtained the inductive ability, but they all neglect semantic information. In this work, we propose an inductive representation learning model TGraiL which considers not only the topological structure but also semantic information. First, distance in the subgraph is used to encode the node’s topological structure. Second, the projection matrix is used to encode the entity type information. Finally, both kinds of information are fused for training to acquire the ultimate vector representation of entities. The experimental results indicate that the model’s performance has been significantly improved compared to the existing baseline models, demonstrating the method’s effectiveness and superiority.

Relphormer: Relational Graph Transformer for Knowledge Graph Representations

Transformers have achieved remarkable performance in widespread fields, including natural language processing, computer vision and graph mining. However, vanilla Transformer architectures have not yielded promising improvements in the Knowledge Graph (KG) representations, where the translational distance paradigm dominates this area. Note that vanilla Transformer architectures struggle to capture the intrinsically heterogeneous structural and semantic information of knowledge graphs. To this end, we propose a new variant of Transformer for knowledge graph representations dubbed Relphormer. Specifically, we introduce Triple2Seq which can dynamically sample contextualized sub-graph sequences as the input to alleviate the heterogeneity issue. We propose a novel structure-enhanced self-attention mechanism to encode the relational information and keep the semantic information within entities and relations. Moreover, we utilize masked knowledge modeling for general knowledge graph representation learning, which can be applied to various KG-based tasks including knowledge graph completion, question answering, and recommendation. Experimental results on six datasets show that Relphormer can obtain better performance compared with baselines.22Code is available in https://github.com/zjunlp/Relphormer.

Joint Representation of Entities and Relations via Graph Attention Networks for Explainable Recommendations

The latest advances in Graph Neural Networks (GNNs), have provided important new ideas for solving the Knowledge Graph (KG) representation problem for recommendation purposes. Although GNNs have an effective graph representation capability, the nonlinear transformations over the layers cause a loss of semantic information and make the generated embeddings hard to explain. In this paper, we investigate the potential of large KGs to perform interpretable recommendation using Graph Attention Networks (GATs). Our goal is to fully exploit the semantic information and preserve inherent knowledge ported in relations by jointly learning low-dimensional embeddings for nodes (i.e., entities) and edges (i.e., properties). Specifically, we feed the original data with additional knowledge from the Linked Open Data (LOD) cloud, and apply GATs to generate a vector representation for each node on the graph. Experiments conducted on three real-world datasets for the top-K recommendation task demonstrate the state-of-the-art performance of the system proposed. In addition to improving predictive performance in terms of precision, recall, and diversity, our approach fully exploits the rich structured information provided by KGs to offer explanation for recommendations.

Advancements in Complex Knowledge Graph Question Answering: A Survey

Complex Question Answering over Knowledge Graph (C-KGQA) seeks to solve complex questions using knowledge graphs. Currently, KGQA systems achieve great success in answering simple questions, while complex questions still present challenging issues. As a result, an increasing number of novel methods have been proposed to remedy this challenge. In this survey, we proposed two mainstream categories of methods for C-KGQA, which are divided according to their use for knowledge graph representation and construction, namely, graph metric (GM)-Based Methods and graph neural network (GNN)-based methods. Additionally, we also acknowledge the influence of ChatGPT, which has prompted further research into utilizing knowledge graphs as a knowledge source to assist in answering complex questions. We also introduced methods based on pre-trained models and knowledge graph joint reasoning. Furthermore, we have compiled research achievements from the past three years to make it easier for researchers with similar interests to obtain state-of-the-art research. Finally, we discussed the resources and evaluation methods for tackling C-KGQA tasks and summarized several research prospects in this field.

Story Shaping: Teaching Agents Human-Like Behavior with Stories

Reward design for reinforcement learning agents can be difficult in situations where one not only wants the agent to achieve some effect in the world but where one also cares about how that effect is achieved. For example, we might wish for an agent to adhere to a tacit understanding of commonsense, align itself to a preference for how to behave for purposes of safety, or taking on a particular role in an interactive game. Storytelling is a mode for communicating tacit procedural knowledge. We introduce a technique, Story Shaping, in which a reinforcement learning agent infers tacit knowledge from an exemplar story of how to accomplish a task and intrinsically rewards itself for performing actions that make its current environment adhere to that of the inferred story world. Specifically, Story Shaping infers a knowledge graph representation of the world state from observations, and also infers a knowledge graph from the exemplar story. An intrinsic reward is generated based on the similarity between the agent's inferred world state graph and the inferred story world graph. We conducted experiments in text-based games requiring commonsense reasoning and shaping the behaviors of agents as virtual game characters.

INK: Knowledge graph representation for efficient and performant rule mining

Semantic rule mining can be used for both deriving task-agnostic or task-specific information within a Knowledge Graph (KG). Underlying logical inferences to summarise the KG or fully interpretable binary classifiers predicting future events are common results of such a rule mining process. The current methods to perform task-agnostic or task-specific semantic rule mining operate, however, a completely different KG representation, making them less suitable to perform both tasks or incorporate each other’s optimizations. This also results in the need to master multiple techniques for both exploring and mining rules within KGs, as well losing time and resources when converting one KG format into another. In this paper, we use INK, a KG representation based on neighbourhood nodes of interest to mine rules for improved decision support. By selecting one or two sets of nodes of interest, the rule miner created on top of the INK representation will either mine task-agnostic or task-specific rules. In both subfields, the INK miner is competitive to the currently state-of-the-art semantic rule miners on 14 different benchmark datasets within multiple domains.

Autonomous complex knowledge mining and graph representation through natural language processing and transfer learning

Regulatory documents play a significant role in securing engineering project quality, standard process management and long-term sustainable developments. With the digitisation of knowledge in the AEC industry, the demand for automated knowledge mining has emerged when confronted with substantial regulations. However, the current interpretation approaches for regulatory documents are still mostly labour-intensive and flawed in complex knowledge. Based on transfer learning (BERT) and natural language processing (e.g., NLP-Syntactic Parsing), this paper proposes a fully automated knowledge mining framework to convert complex knowledge in textual regulations to graph-based knowledge representations. The framework uses a BERT-based engine to extract clauses from regulation documents through fine-tuning with the self-developed domain dataset. A constituent extractor is developed to process the provisions with complex knowledge and extract constituents. A knowledge modelling engine integrates the extracted constituents into a graph-based regulation knowledge model, which can be queried, visualised, and directly applied to downstream applications. The outcome has demonstrated promising performance in complex knowledge mining and knowledge graph modelling based on ISO 19650 case study. This research can effectively convert textual regulation documents to their counterpart regulatory knowledge base, contributing to automated knowledge acquisition and multi-domain knowledge fusion toward regulation digitalization.

Knowledge Graph Representation Learning based on Entity Semantic Distance Classification

Knowledge Graph Representation Learning based on Entity Semantic Distance Classification

Contrastive Multi-Modal Knowledge Graph Representation Learning

Representation learning of knowledge graphs (KGs) aims to embed both entities and relations as vectors in a continuous low-dimensional space, which has facilitated various applications such as link prediction and entity retrieval. Most existing KG embedding methods focus on modeling the structured fact triples independently and ignore the multi-type relations among triples as well as the variety of data types (e.g., texts and images) associated with entities in KGs, and thus fail to capture the complex and multi-modal information that is inherently inside the entity-relation triples. In this paper, we propose a novel approach for knowledge graph embedding named Contrastive Multi-modal Graph Neural Network (CMGNN), which can encapsulate comprehensive features from multi-modal content descriptions of entities and high-order connectivity structures. Specifically, CMGNN first learns entity embeddings from multi-modal content and then contrasts encodings from multi-relational local neighbors and high-order connectivities to obtain latent representations of entities and relations simultaneously. Experimental results demonstrate that CMGNN can effectively model the multi-modalities and multi-type structures in KGs, and significantly outperforms existing state-of-the-art methods on benchmark datasets for the tasks of link prediction and entity classification.