Large Knowledge Graphs Research Articles

Enhancing intention prediction and interpretability in service robots with LLM and KG

The rapid advancement of artificial intelligence has significantly expanded the role of service robots in everyday life. This expansion necessitates the accurate recognition and prediction of human intentions to provide timely and appropriate services. However, existing methods often struggle to perform effectively in complex and unstructured environments. To address this challenge, we propose the Large language model and Knowledge graph based Intention Recognition Framework (LKIRF), which combines large language model (LLM) with knowledge graphs (KG) to enhance the intention recognition capabilities of service robots. Our approach constructs an offline KG from human motion and environmental data and builds an online reasoning graph through real-time interaction, utilizing LLM for interpretation. Experimental results indicate that compared to traditional methods, LKIRF not only improves prediction accuracy across various scenarios but also enhances the transparency and interpretability of the intention reasoning process.

Understanding the structure of knowledge graphs with ABSTAT profiles

While there has been a trend in the last decades for publishing large-scale and highly-interconnected Knowledge Graphs (KGs), their users often get overwhelmed by the task of understanding their content as a result of their size and complexity. Data profiling approaches have been proposed to summarize large KGs into concise and meaningful representations, so that they can be better explored, processed, and managed. Profiles based on schema patterns represent each triple in a KG with its schema-level counterpart, thus covering the entire KG with profiles of considerable size. In this paper, we provide empirical evidence that profiles based on schema patterns, if explored with suitable mechanisms, can be useful to help users understand the content of big and complex KGs. ABSTAT provides concise pattern-based profiles and comes with faceted interfaces for profile exploration. Using this tool we present a user study based on query completion tasks. We demonstrate that users who look at ABSTAT profiles formulate their queries better and faster than users browsing the ontology of the KGs. The latter is a pretty strong baseline considering that many KGs do not even come with a specific ontology to be explored by the users. To the best of our knowledge, this is the first attempt to investigate the impact of profiling techniques on tasks related to knowledge graph understanding with a user study.

Artificial intelligence for literature reviews: opportunities and challenges

This paper presents a comprehensive review of the use of Artificial Intelligence (AI) in Systematic Literature Reviews (SLRs). A SLR is a rigorous and organised methodology that assesses and integrates prior research on a given topic. Numerous tools have been developed to assist and partially automate the SLR process. The increasing role of AI in this field shows great potential in providing more effective support for researchers, moving towards the semi-automatic creation of literature reviews. Our study focuses on how AI techniques are applied in the semi-automation of SLRs, specifically in the screening and extraction phases. We examine 21 leading SLR tools using a framework that combines 23 traditional features with 11 AI features. We also analyse 11 recent tools that leverage large language models for searching the literature and assisting academic writing. Finally, the paper discusses current trends in the field, outlines key research challenges, and suggests directions for future research. We highlight three primary research challenges: integrating advanced AI solutions, such as large language models and knowledge graphs, improving usability, and developing a standardised evaluation framework. We also propose best practices to ensure more robust evaluations in terms of performance, usability, and transparency. Overall, this review offers a detailed overview of AI-enhanced SLR tools for researchers and practitioners, providing a foundation for the development of next-generation AI solutions in this field.

Implementation of Large Language Models and Agricultural Knowledge Graphs for Efficient Plant Disease Detection

This study addresses the challenges of elaeagnus angustifolia disease detection in smart agriculture by developing a detection system that integrates advanced deep learning technologies, including Large Language Models (LLMs), Agricultural Knowledge Graphs (KGs), Graph Neural Networks (GNNs), representation learning, and neural-symbolic reasoning techniques. The system significantly enhances the accuracy and efficiency of disease detection through an innovative graph attention mechanism and optimized loss functions. Experimental results demonstrate that this system significantly outperforms traditional methods across key metrics such as precision, recall, and accuracy, with the graph attention mechanism excelling in all aspects, particularly achieving a precision of 0.94, a recall of 0.92, and an accuracy of 0.93. Furthermore, comparative experiments with various loss functions further validate the effectiveness of the graph attention loss mechanism in enhancing model performance. This research not only advances the application of deep learning in agricultural disease detection theoretically but also provides robust technological tools for disease management and decision support in actual agricultural production, showcasing broad application prospects and profound practical value.

Unifying Large Language Models and Knowledge Graphs: A Roadmap

Unifying Large Language Models and Knowledge Graphs: A Roadmap

Constructing Semantic Summaries Using Embeddings

The increase in the size and complexity of large knowledge graphs now available online has resulted in the emergence of many approaches focusing on enabling the quick exploration of the content of those data sources. Structural non-quotient semantic summaries have been proposed in this direction that involve first selecting the most important nodes and then linking them, trying to extract the most useful subgraph out of the original graph. However, the current state of the art systems use costly centrality measures for identifying the most important nodes, whereas even costlier procedures have been devised for linking the selected nodes. In this paper, we address both those deficiencies by first exploiting embeddings for node selection, and then by meticulously selecting approximate algorithms for node linking. Experiments performed over two real-world big KGs demonstrate that the summaries constructed using our method enjoy better quality. Specifically, the coverage scores obtained were 0.8, 0.81, and 0.81 for DBpedia v3.9 and 0.94 for Wikidata dump 2018, across 20%, 25%, and 30% summary sizes, respectively. Additionally, our method can compute orders of magnitude faster than the state of the art.

Accurate and interpretable drug-drug interaction prediction enabled by knowledge subgraph learning

BackgroundDiscovering potential drug-drug interactions (DDIs) is a long-standing challenge in clinical treatments and drug developments. Recently, deep learning techniques have been developed for DDI prediction. However, they generally require a huge number of samples, while known DDIs are rare.MethodsIn this work, we present KnowDDI, a graph neural network-based method that addresses the above challenge. KnowDDI enhances drug representations by adaptively leveraging rich neighborhood information from large biomedical knowledge graphs. Then, it learns a knowledge subgraph for each drug-pair to interpret the predicted DDI, where each of the edges is associated with a connection strength indicating the importance of a known DDI or resembling strength between a drug-pair whose connection is unknown. Thus, the lack of DDIs is implicitly compensated by the enriched drug representations and propagated drug similarities.ResultsHere we show the evaluation results of KnowDDI on two benchmark DDI datasets. Results show that KnowDDI obtains the state-of-the-art prediction performance with better interpretability. We also find that KnowDDI suffers less than existing works given a sparser knowledge graph. This indicates that the propagated drug similarities play a more important role in compensating for the lack of DDIs when the drug representations are less enriched.ConclusionsKnowDDI nicely combines the efficiency of deep learning techniques and the rich prior knowledge in biomedical knowledge graphs. As an original open-source tool, KnowDDI can help detect possible interactions in a broad range of relevant interaction prediction tasks, such as protein-protein interactions, drug-target interactions and disease-gene interactions, eventually promoting the development of biomedicine and healthcare.

CyberQ: Generating Questions and Answers for Cybersecurity Education Using Knowledge Graph-Augmented LLMs

Building a skilled cybersecurity workforce is paramount to building a safer digital world. However, the diverse skill set, constantly emerging vulnerabilities, and deployment of new cyber threats make learning cybersecurity challenging. Traditional education methods struggle to cope with cybersecurity's rapidly evolving landscape and keep students engaged and motivated. Different studies on students' behaviors show that an interactive mode of education by engaging through a question-answering system or dialoguing is one of the most effective learning methodologies. There is a strong need to create advanced AI-enabled education tools to promote interactive learning in cybersecurity. Unfortunately, there are no publicly available standard question-answer datasets to build such systems for students and novice learners to learn cybersecurity concepts, tools, and techniques. The education course material and online question banks are unstructured and need to be validated and updated by domain experts, which is tedious when done manually. In this paper, we propose CyberGen, a novel unification of large language models (LLMs) and knowledge graphs (KG) to generate the questions and answers for cybersecurity automatically. Augmenting the structured knowledge from knowledge graphs in prompts improves factual reasoning and reduces hallucinations in LLMs. We used the knowledge triples from cybersecurity knowledge graphs (AISecKG) to design prompts for ChatGPT and generate questions and answers using different prompting techniques. Our question-answer dataset, CyberQ, contains around 4k pairs of questions and answers. The domain expert manually evaluated the random samples for consistency and correctness. We train the generative model using the CyberQ dataset for question answering task.

Non-monotonic Generation of Knowledge Paths for Context Understanding

Knowledge graphs can be used to enhance text search and access by augmenting textual content with relevant background knowledge. While many large knowledge graphs are available, using them to make semantic connections between entities mentioned in the textual content remains to be a difficult task. In this work, we therefore introduce contextual path generation (CPG), which refers to the task of generating knowledge paths, contextual path , to explain the semantic connections between entities mentioned in textual documents with given knowledge graph. To perform the CPG task well, one has to address its three challenges, namely, path relevance, incomplete knowledge graph, and path well-formedness. This article designs a two-stage framework comprised of the following: (1) a knowledge-enabled embedding matching and learning-to-rank with multi-head self-attention context extractor to determine a set of context entities relevant to both the query entities and context document, and (2) a non-monotonic path generation method with pretrained transformer to generate high-quality contextual paths. Our experiment results on two real-world datasets show that our best performing CPG model successfully recovers 84.13% of ground truth contextual paths, outperforming the context window baselines. Finally, we demonstrate that the non-monotonic model generates more well-formed paths compared to the monotonic counterpart.

BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights.

In this study, we investigate the potential of large language models (LLMs) to complement biomedical knowledge graphs in the training of semantic models for the biomedical and clinical domains. Drawing on the wealth of the Unified Medical Language System knowledge graph and harnessing cutting-edge LLMs, we propose a new state-of-the-art approach for obtaining high-fidelity representations of biomedical concepts and sentences, consisting of 3 steps: an improved contrastive learning phase, a novel self-distillation phase, and a weight averaging phase. Through rigorous evaluations of diverse downstream tasks, we demonstrate consistent and substantial improvements over the previous state of the art for semantic textual similarity (STS), biomedical concept representation (BCR), and clinically named entity linking, across 15+ datasets. Besides our new state-of-the-art biomedical model for English, we also distill and release a multilingual model compatible with 50+ languages and finetuned on 7 European languages. Many clinical pipelines can benefit from our latest models. Our new multilingual model enables a range of languages to benefit from our advancements in biomedical semantic representation learning, opening a new avenue for bioinformatics researchers around the world. As a result, we hope to see BioLORD-2023 becoming a precious tool for future biomedical applications. In this article, we introduced BioLORD-2023, a state-of-the-art model for STS and BCR designed for the clinical domain.

Improving conversational recommender systems via multi-preference modelling and knowledge-enhanced

Conversational recommender systems (CRS) aim to capture user preferences and provide high-quality recommendations through conversations. Existing CRSs use items in conversations to represent user preferences, and these items usually need to match corresponding entities in external knowledge graphs (KGs). However, current general approaches extract subgraphs from large KGs (e.g., DBpedia) and thus may contain noise from nonrelevant domains and do not cover items in the CRS domain well. In addition, existing CRSs do not focus on hidden semantic preferences in user conversations, nor do they emphasize item attributes. Moreover, the system-generated responses lack diversity and descriptiveness. To address these issues, we propose a novel approach for Improving Conversation Recommendation Systems via Multi-Preference Modelling and Knowledge-Enhanced (MPKE). Specifically, we first customize a higher-quality knowledge graph LMKG for CRS to help the model fully exploit external knowledge to improve its performance. Second, we design a semantic fusion module to augment the semantic connections between external knowledge and dialogue text. Then, we propose a two-stage recommendation mechanism to explore more item attributes and fuse entity and text semantics to enhance user representation. For response generation, we use a word-level KG (i.e., ConceptNet) to enhance the representation of conversation keywords and construct explanation templates with slots to enhance the diversity of system responses. extensive experiments show that our approach outperforms past state-of-the-art methods. the source code will be published at https://github.com/zcy-cqut/MPKE.

AI large models bring great opportunities to reusable design of cad software

With the rapid development of artificial intelligence (AI), large models have achieved significant breakthroughs in general-purpose domains. However, their application in computer-aided design (CAD) software is still in its early stages. Reusable design is crucial for improving efficiency and innovation in CAD systems. This paper reviews progress in rule-based reasoning (RBR) and case-based reasoning (CBR), two prevailing techniques for reusable design. RBR represents expert knowledge as rules but lacks self-learning capabilities. CBR draws on prior cases to solve new problems but relies heavily on surface empirical knowledge. Recent advances in large AI models provide new opportunities to enhance reusable design, thanks to superior language and reasoning abilities. However, adapting large models to effectively leverage CAD-specific design knowledge presents open challenges. To advance progress in this area, this paper analyzes the potential impacts of large models on improving knowledge acquisition, case retrieval, rule representation, and reasoning explain ability for hybrid CBR-RBR systems, and proposes a reusable design framework combing large language models, knowledge graphs, and databases to realize more intelligent and interpretable reuse. This review synthesizes key developments in RBR, CBR, and large AI models, highlighting promising directions for advancing reusable design in CAD software. The integration of reasoning techniques with large models, opening promising new directions for computer-aided engineering enhanced by artificial intelligence, as well as lays the foundation for more efficient, innovative, and sustainable engineering design.

Hierarchical pattern-based complex query of temporal knowledge graph

The complex query is an essential type of knowledge graph query, which aims to deal with the complex relationships among multiple triples involved in query problems. However, the existing query methods often directly search the target entities in large knowledge graphs, resulting in unsatisfactory query results. In addition, most of the methods focus on static knowledge graphs, ignoring the critical semantic information contained in time elements in query questions. Therefore, this paper proposes a complex query model for temporal knowledge graphs based on the hierarchical pattern. Firstly, aiming at the multi-level semantic relations in complex query problems, we use the entity mapping method to generate the corresponding query graphs. This method can accurately express the semantic relations by learning the constraints, such as time and quantity, in the query problem by manual rules. In addition, to reduce the impact of noisy data on query results, a complex query algorithm based on a hierarchical pattern is proposed. By decomposing the large-scale temporal knowledge graph into smaller knowledge sub-graphs, the search range of target results is effectively reduced. Finally, extensive experiments demonstrate that our model has certain advantages in executing complex queries.

Algorithm for the Accelerated Calculation of Conceptual Distances in Large Knowledge Graphs

Conceptual distance refers to the degree of proximity between two concepts within a conceptualization. It is closely related to semantic similarity and relationships, but its measurement strongly depends on the context of the given concepts. DIS-C represents an advancement in the computation of semantic similarity/relationships that is independent of the type of knowledge structure and semantic relations when generating a graph from a knowledge base (ontologies, semantic networks, and hierarchies, among others). This approach determines the semantic similarity between two indirectly connected concepts in an ontology by propagating local distances by applying an algorithm based on the All Pairs Shortest Path (APSP) problem. This process is implemented for each pair of concepts to establish the most effective and efficient paths to connect these concepts. The algorithm identifies the shortest path between concepts, which allows for an inference of the most relevant relationships between them. However, one of the critical issues with this process is computational complexity, combined with the design of APSP algorithms, such as Dijkstra, which is 𝒪n3. This paper studies different alternatives to improve the DIS-C approach by adapting approximation algorithms, focusing on Dijkstra, pruned Dijkstra, and sketch-based methods, to compute the conceptual distance according to the need to scale DIS-C to analyze very large graphs; therefore, reducing the related computational complexity is critical. Tests were performed using different datasets to calculate the conceptual distance when using the original version of DIS-C and when using the influence area of nodes. In situations where time optimization is necessary for generating results, using the original DIS-C model is not the optimal method. Therefore, we propose a simplified version of DIS-C to calculate conceptual distances based on centrality estimation. The obtained results for the simple version of DIS-C indicated that the processing time decreased 2.381 times when compared to the original DIS-C version. Additionally, for both versions of DIS-C (normal and simple), the APSP algorithm decreased the computational cost when using a two-hop coverage-based approach.

A Deep Learning Approach to the Prediction of Drug Side-Effects on Molecular Graphs.

Predicting drug side effects before they occur is a critical task for keeping the number of drug-related hospitalizations low and for improving drug discovery processes. Automatic predictors of side-effects generally are not able to process the structure of the drug, resulting in a loss of information. Graph neural networks have seen great success in recent years, thanks to their ability of exploiting the information conveyed by the graph structure and labels. These models have been used in a wide variety of biological applications, among which the prediction of drug side-effects on a large knowledge graph. Exploiting the molecular graph encoding the structure of the drug represents a novel approach, in which the problem is formulated as a multi-class multi-label graph-focused classification. We developed a methodology to carry out this task, using recurrent Graph Neural Networks, and building a dataset from freely accessible and well established data sources. The results show that our method has an improved classification capability, under many parameters and metrics, with respect to previously available predictors. The method is not ready for clinical tests yet, as the specificity is still below the preliminary 25% threshold. Future efforts will aim at improving this aspect.

Connecting AI: Merging Large Language Models and Knowledge Graph

Connecting AI: Merging Large Language Models and Knowledge Graph

Joint Representation of Entities and Relations via Graph Attention Networks for Explainable Recommendations

The latest advances in Graph Neural Networks (GNNs), have provided important new ideas for solving the Knowledge Graph (KG) representation problem for recommendation purposes. Although GNNs have an effective graph representation capability, the nonlinear transformations over the layers cause a loss of semantic information and make the generated embeddings hard to explain. In this paper, we investigate the potential of large KGs to perform interpretable recommendation using Graph Attention Networks (GATs). Our goal is to fully exploit the semantic information and preserve inherent knowledge ported in relations by jointly learning low-dimensional embeddings for nodes (i.e., entities) and edges (i.e., properties). Specifically, we feed the original data with additional knowledge from the Linked Open Data (LOD) cloud, and apply GATs to generate a vector representation for each node on the graph. Experiments conducted on three real-world datasets for the top-K recommendation task demonstrate the state-of-the-art performance of the system proposed. In addition to improving predictive performance in terms of precision, recall, and diversity, our approach fully exploits the rich structured information provided by KGs to offer explanation for recommendations.

Selective UMLS knowledge infusion for biomedical question answering

One of the artificial intelligence applications in the biomedical field is knowledge-intensive question-answering. As domain expertise is particularly crucial in this field, we propose a method for efficiently infusing biomedical knowledge into pretrained language models, ultimately targeting biomedical question-answering. Transferring all semantics of a large knowledge graph into the entire model requires too many parameters, increasing computational cost and time. We investigate an efficient approach that leverages adapters to inject Unified Medical Language System knowledge into pretrained language models, and we question the need to use all semantics in the knowledge graph. This study focuses on strategies of partitioning knowledge graph and either discarding or merging some for more efficient pretraining. According to the results of three biomedical question answering finetuning datasets, the adapters pretrained on semantically partitioned group showed more efficient performance in terms of evaluation metrics, required parameters, and time. The results also show that discarding groups with fewer concepts is a better direction for small datasets, and merging these groups is better for large dataset. Furthermore, the metric results show a slight improvement, demonstrating that the adapter methodology is rather insensitive to the group formulation.

I-Align: an interpretable knowledge graph alignment model

Knowledge graphs (KGs) are becoming essential resources for many downstream applications. However, their incompleteness may limit their potential. Thus, continuous curation is needed to mitigate this problem. One of the strategies to address this problem is KG alignment, i.e., forming a more complete KG by merging two or more KGs. This paper proposes i-Align, an interpretable KG alignment model. Unlike the existing KG alignment models, i-Align provides an explanation for each alignment prediction while maintaining high alignment performance. Experts can use the explanation to check the correctness of the alignment prediction. Thus, the high quality of a KG can be maintained during the curation process (e.g., the merging process of two KGs). To this end, a novel Transformer-based Graph Encoder (Trans-GE) is proposed as a key component of i-Align for aggregating information from entities’ neighbors (structures). Trans-GE uses Edge-gated Attention that combines the adjacency matrix and the self-attention matrix to learn a gating mechanism to control the information aggregation from the neighboring entities. It also uses historical embeddings, allowing Trans-GE to be trained over mini-batches, or smaller sub-graphs, to address the scalability issue when encoding a large KG. Another component of i-Align is a Transformer encoder for aggregating entities’ attributes. This way, i-Align can generate explanations in the form of a set of the most influential attributes/neighbors based on attention weights. Extensive experiments are conducted to show the power of i-Align. The experiments include several aspects, such as the model’s effectiveness for aligning KGs, the quality of the generated explanations, and its practicality for aligning large KGs. The results show the effectiveness of i-Align in these aspects.

Differentiable learning of rules with constants in knowledge graph

Knowledge reasoning helps overcome the incompleteness of knowledge graphs (KGs) and has significantly contributed to the development of large KGs. Rule mining, one of the key tasks of knowledge reasoning, studies the problem of learning interpretable inference patterns over KGs. Existing rule mining methods mainly focus on learning rules that consist of different relations and variables, restricting the form of rules to be closed path. While rules could be diverse and in order to enrich the forms of rules, we argue that constants should also be considered in the rule mining process. We propose an Elegant Differentiable rUle learning with Constant mEthod (EduCe).11Source code of EduCeis available at https://github.com/yep96/Educe. We propose a constant operator and dynamic weight mechanism, which choose the constants that should be added and decrease the number of parameters, respectively. The model could mine diverse and accurate rules in an efficient way with these modules. The experimental results on several knowledge graph completion benchmarks show that EduCe achieves state-of-the-art link prediction results among differentiable rule mining methods and successfully learns diverse and high-quality rules.