Knowledge Representation Learning Research Articles

Uncovering Multi-step Attacks with Threat Knowledge Graph Reasoning

The rapid advancement of information technologies has significantly intensified the focus on cyberspace security across various sectors. In this evolving landscape, attackers deploy many of techniques- including exploits, weakness identification, and complex multi-step attacks- to gain unauthorized access to systems. Conversely, defenders harness insights from a variety of sources to pinpoint potential threats. Prominent public cybersecurity databases such as the Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK), Common Attack Pattern Enumeration and Classification (CAPEC), Common Vulnerabilities and Exposures (CVE), Common Weakness Enumeration (CWE), and Common Platform Enumeration (CPE) provide extensive data on security entities and their interrelations, playing a pivotal role in enriching the understanding of cybersecurity challenges and assisting in comprehensive defensive analyses. However, the semantic cross-analysis of these databases, crucial for identifying obscure threat patterns, remains underexploited. In this study, we amalgamate data from these disparate sources into a cohesive threat knowledge graph and introduce a novel knowledge representation learning approach, A4CKGE (ATT&CK-CAPEC-CWE-CVE-CPE Knowledge Graph Embedding). This method utilizes advanced structural and textual analytics to predict interactions among security entities such as products, vulnerabilities, weaknesses, and multi-step attack sequences, employing complex attack templates generated through a Large Language Model (LLM). Our extensive experiments demonstrate that this approach significantly outperforms existing state-of-the-art methods in effectively predicting these relationships. The findings validate the efficacy of our threat knowledge graph in unveiling hidden connections, thereby highlighting its potential to strengthen cybersecurity defenses substantially.

Traffic data imputation via knowledge graph-enhanced generative adversarial network

Traffic data imputation is crucial for the reliability and efficiency of intelligent transportation systems (ITSs), forming the foundation for downstream tasks like traffic prediction and management. However, existing deep learning-based imputation methods struggle with two significant challenges: poor performance under high missing data rates and the limited incorporation of external traffic-related factors. To address these challenges, we propose a novel knowledge graph-enhanced generative adversarial network (KG-GAN) for traffic data imputation. Our approach uniquely integrates external knowledge with traffic spatiotemporal dependencies to improve data imputation quality. Specifically, we construct a fine-grained knowledge graph (KG) that differentiates attributes and relationships of external factors such as points of interest (POI) and weather conditions, facilitating more robust knowledge representation learning. We then introduce a knowledge-aware embedding cell (EM-cell) that merges traffic data with these learned external representations, providing richer inputs for the spatiotemporal GAN. Extensive experiments on a large-scale real-world traffic dataset demonstrate that KG-GAN significantly outperforms state-of-the-art methods under various missing data scenarios. Additionally, ablation studies confirm the superior performance gained from incorporating external knowledge, underscoring the importance of this approach in addressing complex missing data patterns.

Promoting Machine Abilities of Discovering and Utilizing Knowledge in a Unified Zero-shot Learning Paradigm

Knowledge discovery and utilization are two essential cognitive processes that enable humans to understand the world and extract new insights from their surroundings. These processes have motivated machine learning studies, particularly zero-shot learning (ZS), which seeks to identify unseen concepts through the use of side information. Previous ZS studies primarily focused on utilizing existing knowledge to infer unseen events, yet they overlook the crucial process of knowledge discovery and the integrated modeling of these knowledge-aware processes. In this study, we present a comprehensive ZS learning approach that explores and evaluates the machine's abilities of discovering and utilizing knowledge. More specifically, to emulate human-like knowledge discovery and utilization processes, we propose a novel visual-aware ZS knowledge graph completion task for evaluation, incorporating a traditional ZS image classification task. Technically, we develop a unified ZS learning paradigm named Cognitive Learner (CoLa) to foster the two knowledge-aware abilities. Including a knowledge representation learning (KRL) module and a knowledge adaptation (KA) module, CoLa adapts well to the two specified tasks with the corresponding data. Extensive experiments on large-scale datasets demonstrate CoLa models’ outstanding performance over compared methods in the two ZS tasks, illustrating their superior ability of discovering and utilizing knowledge.

Knowledge Representation Learning Method Based on Semantic Enhancement of External Information

Background: Knowledge representation learning aims at mapping entity and relational data in knowledge graphs to a low-dimensional space in the form of vectors. The existing work has mainly focused on structured information representation of triples or introducing only one additional kind of information, which has large limitations and reduces the representation efficiency. Objective: This study aims to combine entity description information and textual relationship description information with triadic structure information, and then use the linear mapping method to linearly transform the structure vector and text vector to obtain the joint representation vector. Methods: A knowledge representation learning (DRKRL) model that fuses external information for semantic enhancement is proposed, which combines entity descriptions and textual relations with a triadic structure. For entity descriptions, a vector representation is performed using a bi-directional long- and short-term memory network (Bi-LSTM) model and an attention mechanism. For the textual relations, a convolutional neural network is used to vectorially encode the relations between entities, and then an attention mechanism is used to obtain valuable information as complementary information to the triad. Results: Link prediction and triadic group classification experiments were conducted on the FB15K, FB15K-237, WN18, WN18RR, and NELL-995 datasets. Theoretical analysis and experimental results show that the DRKRL model proposed in this paper has higher accuracy and efficiency compared with existing models. Conclusion: Combining entity description information and textual relationship description information with triadic structure information can make the model have better performance and effectively improve the knowledge representation learning ability.

Meta-collaboration-based semantic contrast for inductive knowledge representation learning

Inductive knowledge representation learning aims at effectively representing new entities in emerging knowledge graphs based on current ones, providing positive support for reasoning facts with newly emerging entities. It requires a scheme that can derive the instant knowledge to generate rational representations for new entities. Recently, some approaches have been developed for it by capturing logical rules between entities, mining structural information of KGs, or learning multiple structural patterns of entities. However, there are two main challenges that they have yet to overcome. The first is the structural bias-induced semantic ambiguity, where new entities are typically represented in a manner akin to those with analogous structures, during which the identical structures are emphasized while the differences are overlooked. We refer to this as the structural bias, which will lead to the generated representations of new entities being semantically ambiguous. The second is the potential sparsity of new entities. The presence of new entities with low frequencies brings about the insufficient representations for these new entities even training a model from scratch with substantial costs. To this end, we propose a Meta-Collaboration-based Semantic Contrast (MCSC) framework with the aim of addressing both issues above. Following the current meta-task formulation scheme, we first acquire the relation-specific knowledge to produce entity representations through modeling relations as feedback. Then, a multi-layer graph neural network module is developed to aggregate sufficient neighbor information to update entity representations. In the final, a collaborative semantic contrast module is designed on both support and query sets to respectively overcome the structural bias-induced semantic ambiguity and the potential sparsity of new entities challenges. Such a meta-collaboration-based semantic contrast procedure transcends current meta-learning paradigms in inductive knowledge representation learning that fail to learn semantically discriminative entity representations. Experimental results of the inductive link prediction task on 12 benchmark datasets demonstrate the superiority of MCSC over SOTA baselines.

A joint knowledge representation learning of sentence vectors weighting and primary neighbor constraints

A joint knowledge representation learning of sentence vectors weighting and primary neighbor constraints

A knowledge graphs representation method based on IsA relation modeling

Knowledge graph representation learning, which is able to support further knowledge computation and reasoning, has drawn much research attention in the field of artificial intelligence in recent years. Due to data sparsity, traditional knowledge graph methods focusing on the observed facts have poor performance. Therefore, researchers introduce auxiliary information to tackle this issue. As a kind of prior knowledge containing rich semantics, IsA relation could be used to boost model performance. We propose a knowledge representation learning method combined with isA relation modeling. Specifically, we model two features of isA relation—transitivity and antisymmetry. The transitivity is modeled by the projection invariance of vectors, and antisymmetry is modeled by partial order constraints. We use RotatE as our base model which could also be replaced by other knowledge embedding model with higher dimension. Finally, experimental results of link prediction and triple classification task on two datasets verify the proposed model’s effectiveness outperforming baseline models.

Knowledge is power: Open-world knowledge representation learning for knowledge-based visual reasoning

Knowledge-based visual reasoning requires the ability to associate outside knowledge that is not present in a given image for cross-modal visual understanding. Two deficiencies of the existing approaches are that (1) they only employ or construct elementary and explicit but superficial knowledge graphs while lacking complex and implicit but indispensable cross-modal knowledge for visual reasoning, and (2) they also cannot reason new/unseen images or questions in open environments and are often violated in real-world applications. How to represent and leverage tacit multimodal knowledge for open-world visual reasoning scenarios has been less studied. In this paper, we propose a novel open-world knowledge representation learning method to not only construct implicit knowledge representations from the given images and their questions but also enable knowledge transfer from a known given scene to an unknown scene for answer prediction. Extensive experiments conducted on six benchmarks demonstrate the superiority of our approach over other state-of-the-art methods. We apply our approach to other visual reasoning tasks, and the experimental results show that our approach, with its good performance, can support related reasoning applications.

A Condition Knowledge Representation and Feedback Learning Framework for Dynamic Optimization of Integrated Energy Systems.

An optimal energy scheduling strategy for integrated energy systems (IESs) can effectively improve the energy utilization efficiency and reduce carbon emissions. Due to the large-scale state space of IES caused by uncertain factors, it would be beneficial for the model training process to formulate a reasonable state-space representation. Thus, a condition knowledge representation and feedback learning framework based on contrastive reinforcement learning is designed in this study. Considering that different state conditions would bring inconsistent daily economic costs, a dynamic optimization model based on deterministic deep policy gradient is established, so that the condition samples can be partitioned according to the preoptimized daily costs. In order to represent the overall conditions on a daily basis and constrain the uncertain states in the IES environment, the state-space representation is constructed by a contrastive network considering the time dependence of variables. A Monte-Carlo policy gradient-based learning architecture is further proposed to optimize the condition partition and improve the policy learning performance. To verify the effectiveness of the proposed method, typical load operation scenarios of an IES are used in our simulations. The human experience strategies and state-of-the-art approaches are selected for comparisons. The results validate the advantages of the proposed approach in terms of cost effectiveness and ability to adapt in uncertain environments.

A multi-view heterogeneous knowledge learning method for personalized POI recommendation

Existing POI recommendation methods often fail to capture the fine-grained preferences of users and face the challenge of modeling multiple relationships. Moreover, knowledge graph-based recommendation methods are limited in storing dynamic user trajectories, making them unsuitable for POI recommendation scenarios. In this paper, we propose a Multi-View Heterogeneous Knowledge learning model that utilizes techniques for heterogeneous knowledge representation learning and multi-view context modeling. Our model comprehensively models user preferences and the relationships between users and POIs by utilizing information from users’ visiting sequences and POI attributes knowledge graph. Specifically, we design a heterogeneous knowledge embedding method to learn the representation of users and POIs using POI attribute knowledge and users’ visiting sequences. Additionally, we constructed a user trajectory similarity graph and a POI attribute similarity graph to explore potential relations between users and between POIs. The former measures the similarity of user behaviors based on user visit sequences, and the latter quantifies the similarity between different POIs through a novel feature mapping method. Finally, we propose a multi-view hybrid learning method that combines unsupervised and supervised learning paradigms to model complex relationships, improving the overall recommendation performance. Extensive experiments on real-world datasets validate the effectiveness of our method.

A Novel Method for Boosting Knowledge Representation Learning in Entity Alignment through Triple Confidence

A Novel Method for Boosting Knowledge Representation Learning in Entity Alignment through Triple Confidence

Incorporating environmental knowledge embedding and spatial-temporal graph attention networks for inland vessel traffic flow prediction

Accurate prediction of vessel traffic flow is crucial for maritime regulatory authorities and transportation planners. However, existing methods for inland vessel traffic flow prediction often overlook spatial correlation and environmental influences, leading to suboptimal accuracy. To address this issue, we propose an innovative model that incorporates environmental knowledge embedding and a spatial-temporal information extraction module. Our approach involves constructing a vessel traffic knowledge graph, embedding traffic flow through knowledge representation learning. The spatial-temporal information extraction module is leveraged to analyze inherent periodicity and external spatial relationships in vessel traffic flow. Extensive experiments on real-world datasets demonstrate that our approach significantly enhances predictive accuracy. In comparison to the second-ranked model, our approach achieves a decrease of 0.46 in mean absolute error, a decrease of 0.64 in root mean squared error, an increase of 3.06% in accuracy, and an increase of 0.07 in R-squared. Furthermore, our approach excels in upstream, downstream and long-term prediction, and displays robustness in handling noisy data.

A Survey of Knowledge Enhanced Pre-trained Language Models

Pre-trained language models learn informative word representations on a large-scale text corpus through self-supervised learning, which has achieved promising performance in fields of natural language processing (NLP) after fine-tuning. These models, however, suffer from poor robustness and lack of interpretability. We refer to pre-trained language models with knowledge injection as knowledge-enhanced pre-trained language models (KEPLMs). These models demonstrate deep understanding and logical reasoning and introduce interpretability. In this survey, we provide a comprehensive overview of KEPLMs in NLP. We first discuss the advancements in pre-trained language models and knowledge representation learning. Then we systematically categorize existing KEPLMs from three different perspectives. Finally, we outline some potential directions of KEPLMs for future research.

Toward a framework of extracting typical machining process routines based on knowledge representation learning

With the application of CAPP systems, manufacturing enterprises have accumulated a large volume of process data of parts, which are important strategic resources. Deeply mining the value hidden in these process data to improve the reuse rate of process knowledge is the key to realize the intellectualization of process planning. This paper builds a novel framework of extracting typical machining process routines (TMPRs) based on knowledge representation learning. First, two graph models, which describe machining processes including the relationships among “part-material-process-machine tool-cutting tool”, are established to present the semantic meaning of process data through the connectivity of graph structures in a direct way. One model conforms to the standard process routes, while the other makes some modifications to make the node “parts” have more in-links and out-links. Then, TransD algorithm is used to map the entities and relations based on the two models into dense low-dimensional real-valued spaces, and the better model is determined. After that, the vectors representing the parts of the better model are clustered, and the results show that the parts with a similar operation sequence and processed by the same cutting tools and machine tools can be clustered into the same cluster. This method breaks through the singleness and limitation of the traditional classification depending on the features of parts. Furthermore, TMPRs are extracted from the cluster results to verify the expression ability of semantic meaning of these vectors. Finally, we package the main functions of the research results into corresponding software modules. Compared with the existing process reuse methods, the proposed framework is much more efficient and extensible.

CoolGust: knowledge representation learning with commonsense knowledge guidelines and constraints

CoolGust: knowledge representation learning with commonsense knowledge guidelines and constraints

A knowledge representation learning model based on relation rotation in two-dimensional Minkowski space

A knowledge representation learning model based on relation rotation in two-dimensional Minkowski space

Theoretical Advancement of Xi Jinping’s Socialist Thought with Chinese Characteristics in the New Era under the Construction of Knowledge Map

Abstract This paper discusses the theory and practice of constructing knowledge graph under the guidance of Xi Jinping’s socialist thought with Chinese characteristics in the new era. The study first adopts the Transformer technique based on the BERT model for effectively extracting entity relationships. Then, the article proposes an entity alignment method based on Neural Tensor Network, which enhances the semantic and structural information utilization of entity embedding vectors through optimized NTN and joint knowledge representation learning. In addition, the study employs a knowledge graph link prediction method that combines semantics and paths to fill in missing relations, and applies probabilistic soft logic to solve the inference problem of non-deterministic knowledge. Ultimately, the effectiveness of the constructed knowledge graph is demonstrated through empirical analysis of Xi Jinping’s theoretical dataset of socialist thought with Chinese characteristics in the new era. The results show that the method is more effective in dealing with polysemous words and participle errors than traditional methods, improving the data structuring level. For example, the accuracy of entity alignment is significantly better than the baseline algorithm on different datasets, e.g., on the DBP-WE dataset, its Hits@1, Hits@10 and MRR values are higher than those of the baseline by 4.01-29.49, 1.99-31.05, and 0.036-0.289, respectively. The method of this paper is remarkably effective in improving the performance of the entity alignment task, which is helpful in understanding and applying the Xi Jinping’s thought of socialism with Chinese characteristics in the new era provides new theoretical perspectives and practical methods.

A Study of Minority Culture Outreach Translation Based on Cognitive Mapping Construction

Abstract In this paper, we first construct a cognitive neural network and use the knowledge representation learning technique for knowledge complementation and relation extraction to semantically match head-tail entity pairs with relations and accurately model the semantic relationships between entities and relations. Then, the vectors of each entity are mapped onto the hyperplane of a given relation by the cognitive mapping translation model, and vector translation operations are performed on the hyperplane. Finally, this paper’s translation model is compared with other models to compare the semantic representation translation, task processing function, and model complexity of each model so as to verify the superiority of this paper’s model for minority culture outreach translation. The results show that this paper’s method increases 3.4 and 1.2 BLEU values on average in the four translation tasks compared to the Imagination model and VAG model, respectively. When the number of iterations is 40, the MRR value of this paper’s model is 0.345, which is at least 0.1-0.5 more than the other models, thus indicating that this paper’s model has a better performance in outreach translation.

Using TransR to enhance drug repurposing knowledge graph for COVID-19 and its complications

MotivationThe COVID-19 pandemic has been spreading globally for four years, yet specific drugs that effectively suppress the virus remain elusive. Furthermore, the emergence of complications associated with COVID-19 presents significant challenges, making the development of therapeutics for COVID-19 and its complications an urgent task. However, traditional drug development processes are time-consuming. Drug repurposing, which involves identifying new therapeutic applications for existing drugs, presents a viable alternative. ResultIn this study, we construct a knowledge graph by retrieving information on genes, drugs, and diseases from databases such as DRUGBANK and GNBR. Next, we employ the TransR knowledge representation learning approach to embed entities and relationships into the knowledge graph. Subsequently, we train the knowledge graph using a graph neural network model based on TransR scoring. This trained knowledge graph is then utilized to predict drugs for the treatment of COVID-19 and its complications. Based on experimental results, we have identified 15 drugs out of the top 30 with the highest success rates associated with treating COVID-19 and its complications. Notably, out of these 15 drugs, 10 specifically aimed at treating COVID-19, such as Torcetrapib and Tocopherol, has not been previously identified in the knowledge graph. This finding highlights the potential of our model in aiding healthcare professionals in drug development and research related to this disease.

The impact of artificial intelligence systems and technology on the sustainability of the quality of financial reports

The research aims to identify the concept of sustainability in financial reports and the arguments against and in favor of it when applied in business companies. Identify the factors affecting the process of sustaining the quality of financial reports Identify artificial intelligence techniques and know the expected impact of their application on business companies. Clarify the role of artificial intelligence techniques on the sustainability of the quality of financial reports. To achieve the objectives of the research The researcher organized a questionnaire form that included the main axes of artificial intelligence and the extent of its impact on the sustainability of the quality of financial reports, and the research reached several results, the most important of which is that artificial intelligence is one of the most important technologies that have the impact of the value and usefulness of the information shown by financial reports in a fast way and with a distinctive presentation, which makes it more expensive and enhances the sustainability of the quality of financial reports. The application of artificial intelligence systems and techniques is important and inevitable in the field of accounting, which will lead to high quality and continuity in the validity of information, which in turn will lead to the sustainability and quality of financial reports. The research recommended several recommendations, the most important of which is to enhance the use and application of artificial intelligence systems and techniques in economic units to raise the efficiency of accounting operations, which in turn enhances the sustainability of the quality of financial reports. Economic units should provide adequate assistance to the components of artificial intelligence from expert systems, knowledge representation and automatic learning in acquiring knowledge stored in databases that help raise the efficiency of information and the sustainability of the quality of financial reports.