Graph Embedding Methods Research Articles

Review of heterogeneous graph embedding methods based on deep learning techniques and comparing their efficiency in node classification

Review of heterogeneous graph embedding methods based on deep learning techniques and comparing their efficiency in node classification

SimRE: Simple contrastive learning with soft logical rule for knowledge graph embedding

Knowledge graphs serve as a pivotal framework for the structured representation of information regarding entities and relations. However, in the real world, these knowledge graphs are often incomplete and harboring missing facts. Knowledge graph completion (KGC) has emerged as a central research focus, entailing the automated prediction of these missing facts and garnering substantial scholarly attention in recent years. Text-based knowledge graph embedding methods have demonstrated considerable potential for tackling the challenges associated with KGC by employing pre-trained language models. However, their limitation lies in the lack of logical features, which constrains the efficacy of capturing intricate patterns within knowledge graphs. This paper proposed SimRE, a straightforward contrastive learning framework augmented with soft logic rules. SimRE introduces a self-supervised framework that leverages the input rule bodies to predict the corresponding rule heads through a contrastive objective. We introduced two rule sampling techniques to enhance the efficiency and accuracy of the model: in-batch rule negatives and pre-batch rule negatives. SimRE employs a simple method for integrating logical features with the text-based model. The experimental results on benchmark datasets demonstrate that the proposed approach outperforms state-of-the-art methods.

DynG2G: An Efficient Stochastic Graph Embedding Method for Temporal Graphs

Dynamic graph embedding has gained great attention recently due to its capability of learning low-dimensional and meaningful graph representations for complex temporal graphs with high accuracy. However, recent advances mostly focus on learning node embeddings as deterministic "vectors" for static graphs, hence disregarding the key graph temporal dynamics and the evolving uncertainties associated with node embedding in the latent space. In this work, we propose an efficient stochastic dynamic graph embedding method (DynG2G) that applies an inductive feedforward encoder trained with node triplet energy-based ranking loss. Every node per timestamp is encoded as a time-dependent probabilistic multivariate Gaussian distribution in the latent space, and, hence, we are able to quantify the node embedding uncertainty on-the-fly. We have considered eight different benchmarks that represent diversity in size (from 96 nodes to 87 626 and from 13 398 edges to 4 870 863) as well as diversity in dynamics, from slowly changing temporal evolution to rapidly varying multirate dynamics. We demonstrate through extensive experiments based on these eight dynamic graph benchmarks that DynG2G achieves new state-of-the-art performance in capturing the underlying temporal node embeddings. We also demonstrate that DynG2G can simultaneously predict the evolving node embedding uncertainty, which plays a crucial role in quantifying the intrinsic dimensionality of the dynamical system over time. In particular, we obtain a "universal" relation of the optimal embedding dimension, Lo , versus the effective dimensionality of uncertainty, Du , and infer that Lo=Du for all cases. This, in turn, implies that the uncertainty quantification approach we employ in the DynG2G algorithm correctly captures the intrinsic dimensionality of the dynamics of such evolving graphs despite the diverse nature and composition of the graphs at each timestamp. In addition, this L0 - Du correlation provides a clear path to selecting adaptively the optimum embedding size at each timestamp by setting L ≥ Du .

Digging the Underlying Layers of Intelligent Transportation System Networks by Mobile Crowdsourcing

Understanding the underlying topologies of networks is very important for improving their performance and security, especially in Intelligent Transportation Systems (ITS). Anonymous router identification recognizes and merges anonymous interfaces belonging to the same router in the traceroute measurement data, which is regarded as one of the critical issues for recovering the router-level topologies of networks. In this paper, we studied the problem of recovering the router-level topologies of networks with mobile crowdsourcing and proposed a method based on graph embedding (called <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GE)</monospace> for accurate identification of anonymous network routers from traceroute measurement data. We first use the graph embedding method to capture the connection characteristics of different interfaces of anonymous routers from the traceroute data and then, evaluate the distances between interfaces based on the extracted connection characteristics. Finally, we identify the interfaces that belong to the same anonymous router by using a heuristic maximum likelihood estimation algorithm that we have developed. Compared to the existing methods, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GE</monospace> is more felxible in ITS because it identifies anonymous network routers without requiring manually predefined subgraphs (or patterns). In the evaluation, we simulated a variety of scenarios to generate the measurement data and compared <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GE</monospace> to two state-of-the-art methods (the neighborhood matching method and the graph induction-based method). The results suggest that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GE</monospace> can significantly improve the accuracy of anonymous router identification (with an average improvement of over 20%) at a slight identification rate loss (less than 4%).

Prediction of Drug-Disease Associations Based on Multi-Kernel Deep Learning Method in Heterogeneous Graph Embedding.

Computational drug repositioning can identify potential associations between drugs and diseases. This technology has been shown to be effective in accelerating drug development and reducing experimental costs. Although there has been plenty of research for this task, existing methods are deficient in utilizing complex relationships among biological entities, which may not be conducive to subsequent simulation of drug treatment processes. In this article, we propose a heterogeneous graph embedding method called HMLKGAT to infer novel potential drugs for diseases. More specifically, we first construct a heterogeneous information network by combining drug-disease, drug-protein and disease-protein biological networks. Then, a multi-layer graph attention model is utilized to capture the complex associations in the network to derive representations for drugs and diseases. Finally, to maintain the relationship of nodes in different feature spaces, we propose a multi-kernel learning method to transform and combine the representations. Experimental results demonstrate that HMLKGAT outperforms six state-of-the-art methods in drug-related disease prediction, and case studies of five classical drugs further demonstrate the effectiveness of HMLKGAT.

RoCS: Knowledge Graph Embedding Based on Joint Cosine Similarity

Knowledge graphs usually have many missing links, and predicting the relationships between entities has become a hot research topic in recent years. Knowledge graph embedding research maps entities and relations to a low-dimensional continuous space representation to predict links between entities. The present research shows that the key to the knowledge graph embedding approach is the design of scoring functions. According to the scoring function, knowledge graph embedding methods can be classified into dot product models and distance models. We find that the triple scores obtained using the dot product model or the distance model were unbounded, which leads to large variance. In this paper, we propose RotatE Cosine Similarity (RoCS), a method to compute the joint cosine similarity of complex vectors as a scoring function to make the triple scores bounded. Our approach combines the rotational properties of the complex vector embedding model RotatE to model complex relational patterns. The experimental results demonstrate that the newly introduced RoCS yields substantial enhancements compared to RotatE across various knowledge graph benchmarks, improving up to 4.0% in hits at 1 (Hits@1) on WN18RR and improving up to 3.3% in Hits@1 on FB15K-237. Meanwhile, our method achieves some new state-of-the-art (SOTA), including Hits@3 of 95.6%, Hits@10 of 96.4% on WN18, and mean reciprocal rank (MRR) of 48.9% and Hits@1 of 44.5% on WN18RR.

The Full m Index Sets of P2×Pn

Shiu and Kwong (2008) studied the full friendly index set of P2×Pn, which only addressed the cases where m=0 or 1. In this paper, we significantly extend their work by determining the full m index set MP2×Pn for all values of m. Our key approach is to utilize graph embedding and recursion methods to deduce MP2×Pn for general m. In particular, we embed small graphs like C4 and K2 into P2×Pn and apply recursive techniques to prove the main results. This work expands the scope of previous graph labeling studies and provides new insights into determining the full m index set of product graphs. Given the broad range of applications for labeled graphs, this research can potentially impact fields like coding theory, communication network design, and more.

Bert-based graph unlinked embedding for sentiment analysis

Numerous graph neural network (GNN) models have been used for sentiment analysis in recent years. Nevertheless, addressing the issue of over-smoothing in GNNs for node representation and finding more effective ways to learn both global and local information within the graph structure, while improving model efficiency for scalability to large text sentiment corpora, remains a challenge. To tackle these issues, we propose a novel Bert-based unlinked graph embedding (BUGE) model for sentiment analysis. Initially, the model constructs a comprehensive text sentiment heterogeneous graph that more effectively captures global co-occurrence information between words. Next, by using specific sampling strategies, it efficiently preserves both global and local information within the graph structure, enabling nodes to receive more feature information. During the representation learning process, BUGE relies solely on attention mechanisms, without using graph convolutions or aggregation operators, thus avoiding the over-smoothing problem associated with node aggregation. This enhances model training efficiency and reduces memory storage requirements. Extensive experimental results and evaluations demonstrate that the adopted Bert-based unlinked graph embedding method is highly effective for sentiment analysis, especially when applied to large text sentiment corpora.

Reorganizing heterogeneous information from host–microbe interaction reveals innate associations among samples

AbstractThe information on host–microbe interactions contained in the operational taxonomic unit (OTU) abundance table can serve as a clue to understanding the biological traits of OTUs and samples. Some studies have inferred the taxonomies or functions of OTUs by constructing co‐occurrence networks, but co‐occurrence networks can only encompass a small fraction of all OTUs due to the high sparsity of the OTU table. There is a lack of studies that intensively explore and use the information on sample‐OTU interactions. This study constructed a sample‐OTU heterogeneous information network and represented the nodes in the network through the heterogeneous graph embedding method to form the OTU space and sample space. Taking advantage of the represented OTU and sample vectors combined with the original OTU abundance information, an Integrated Model of Embedded Taxonomies and Abundance (IMETA) was proposed for predicting sample attributes, such as phenotypes and individual diet habits. Both the OTU space and sample space contain reasonable biological or medical semantic information, and the IMETA using embedded OTU and sample vectors can have stable and good performance in the sample classification tasks. This suggests that the embedding representation based on the sample‐OTU heterogeneous information network can provide more useful information for understanding microbiome samples. This study conducted quantified representations of the biological characteristics within the OTUs and samples, which is a good attempt to increase the utilization rate of information in the OTU abundance table, and it promotes a deeper understanding of the underlying knowledge of human microbiome.

GE-DDRL: Graph Embedding and Deep Distributional Reinforcement Learning for Reliable Shortest Path: A Universal and Scale Free Solution

This paper studies the reliable shortest path (RSP) problem in stochastic transportation networks. State-of-the-art RSP solutions usually target one specific RSP problem; moreover, the corresponding algorithm’s computational complexity scales at least linearly with the size of the underlying transportation network. While in this paper, we propose a graph embedding and deep distributional reinforcement learning (GE-DDRL) method, which serves as a universal and scale-free solution to the RSP problem. GE-DDRL uses deep distributional reinforcement learning (DDRL) to estimate the full travel-time distribution of a given routing policy, and improves the given routing policy with the generalized policy iteration (GPI) scheme. Further, in order to achieve the generalization ability to new destination nodes, we employ one of the canonical graph embedding techniques (Skip-Gram) to compress the nodes’ representation into <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$d$</tex-math> </inline-formula> -dimensional real-valued vectors. With the properly compressed node features, GE-DDRL is able to generalize its estimation of the routing policy’s travel-time distribution to untrained destination nodes, and hence achieve the ‘all-to-all’ navigation functionality. To the best of our knowledge, GE-DDRL serves as the first RSP planner, which applies simultaneously to almost all RSP objectives and in the meanwhile, is scale free with the size of the transportation network in terms of the online decision-making time and memory complexity. Experimental results and comparisons with state of the arts show the efficacy and efficiency of GE-DDRL in a range of transportation networks.

Multi-ontology embeddings approach on human-aligned multi-ontologies representation for gene-disease associations prediction

ObjectivesKnowledge graphs and ontologies in the biomedical domain provide rich contextual knowledge for a variety of challenges. Employing that for knowledge-driven NLP tasks such as gene-disease association prediction represents a promising way to increase the predictive power of a model. MethodsWe investigated the power of infusing the embedding of two aligned ontologies as prior knowledge to the NLP models. We evaluated the performance of different models on some large-scale gene-disease association datasets and compared it with a model without incorporating contextualized knowledge (BERT). ResultsThe experiments demonstrated that the knowledge-infused model slightly outperforms BERT by creating a small number of bridges. Thus, indicating that incorporating cross-references across ontologies can enhance the performance of base models without the need for more complex and costly training. However, further research is needed to explore the generalizability of the model. We expected that adding more bridges would bring further improvement based on the trend we observed in the experiments. In addition, the use of state-of-the-art knowledge graph embedding methods on a joint graph from connecting OGG and DOID with bridges also yielded promising results. ConclusionOur work shows that allowing language models to leverage structured knowledge from ontologies does come with clear advantages in the performance. Besides, the annotation stage brought out in this paper is constrained in reasonable complexity.

Attention-based exploitation and exploration strategy for multi-hop knowledge graph reasoning

Knowledge Graphs (KGs) typically suffer from incompleteness. A popular approach to solve this problem is multi-hop reasoning through Reinforcement Learning (RL) framework, which is an explainable and effective model to predict missing links in KGs. However, many previous RL-based models use the scoring function of the pre-trained Knowledge Graph Embedding (KGE) methods as the reward function, which will lead to the performance of the model be limited to the KGE methods. Moreover, the agent may reason a meaningless path if it cannot distinguish the different aspect of each entity in different triples. To solve both problems, we propose a multi-hop reasoning model named Ae2KGR, by applying two novel strategies: attention-based exploitation and attention-based exploration. The attention-based exploitation strategy incorporates historical and query information with the neighborhood of the current entity, and dynamically updates the current state during reasoning process to assign different semantic information to the entity to distinguish different triples. The attention-based exploration strategy designs a novel policy network and reward function to dynamically make decisions based on the constantly changing state. Extensive experiments on three standard datasets confirm the effectiveness of our innovations, and the performance of our proposed Ae2KGR is significantly improved compared to the state-of-the-art methods.

HPRE: Leveraging hierarchy-aware paired relation vectors for knowledge graph embedding

Knowledge graphs exhibit a typical hierarchical structure and find extensive applications in various artificial intelligence domains. However, large-scale knowledge graphs need to be completed, which limits the performance of knowledge graphs in downstream tasks. Knowledge graph embedding methods have emerged as a primary solution to enhance knowledge graph completeness. These methods aim to represent entities and relations as low-dimensional vectors, focusing on handling relation patterns and multi-relation types. Researchers need to pay more attention to the crucial feature of hierarchical relationships in real-world knowledge graphs. We propose a novel knowledge graph embedding model called Hierarchy-Aware Paired Relation Vectors Knowledge Graph Embedding (HPRE) to bridge this gap. By leveraging the power of 2D coordinates, HPRE adeptly model relation patterns, multi-relation types, and hierarchical features in the knowledge graph. Specifically, HPRE employs paired relation vectors to capture the distinct characteristics of head and tail entities, facilitating a better fit for relational patterns and multi-relation scenarios. Additionally, HPRE employs angular coordinates to differentiate entities at various levels of the hierarchy, effectively representing the hierarchical nature of the knowledge graph. The experimental results show that the HPRE model can effectively learn the hierarchical features of the knowledge graph and achieve state-of-the-art experimental results on multiple real-world datasets for the link prediction task.

RHGNN: Fake reviewer detection based on reinforced heterogeneous graph neural networks

In e-commerce, fake reviewers frequently post fake reviews to mislead consumers into making unwise shopping decisions, seriously affecting customers’ benefits. Graph neural networks (GNNs) have been widely cultivated in fake reviewer detection in recent years, but they exhibit two main limitations. Firstly, most approaches model reviewer detection with a homogeneous graph, ignoring the interdependent clues among reviewers, reviews, and products, making them ineffective against stealthy fake reviewers. Secondly, few works noticed the camouflage behavior of fake reviewers, i.e., fake reviewers disguise themselves as authentic ones by establishing links with genuine reviewers, which extremely weakens the suspiciousness of fake reviewers and undermines detection performance. This paper proposes RHGNN, a robust fake reviewers detection framework based on Reinforced Heterogeneous Graph Neural Networks. Particularly, we first model fake reviewer detection with a heterogeneous graph consisting of reviews, reviewers, and products, to capture stealthy fraud clues by investigating their heterogeneous interactive relations. Subsequently, we present a reinforced neighbor selector (RNS) to sample and aggregate top−k informative neighbors while filtering out the camouflaged relationships. Finally, we propose a self-supervised heterogeneous graph embedding method based on mutual information to learn the discriminative features of fake reviewers. Experimental results show that RHGNN demonstrates promising and robust performance, especially in detecting well-disguised fake reviewers.

Graphfool: Targeted Label Adversarial Attack on Graph Embedding

Graph embedding learns low-dimensional representations for nodes or edges on the graph, which is widely applied in many real-world applications. Excessive graph mining promotes the research of attack methods on graph embedding. Most attack methods generate perturbations that maximize the deviation of the prediction confidence. They are difficult to accurately misclassify the instances into the target label, and the nonminimized perturbations are more easily detected by defense methods. To address these problems, we propose Graphfool, a novel targeted label adversarial attack on graph embedding. It can generate adversarial graphs to attack graph embedding methods via classification boundary and gradient information in the target graph embedding method. Graphfool first estimates the classification boundaries of different categories. Then, it calculates the minimum perturbation matrix to misclassify the attacked node according to the target classification boundary. Finally, the adjacency matrix is modified according to the maximum absolute value of the perturbation matrix. Extensive experiments demonstrate that Graphfool achieves the state-of-the-art attack performance with minimum perturbations. Besides, the possible defense experiments further prove that the perturbations generated by Graphfool are more imperceptible.

Multi-Information Preprocessing Event Extraction With BiLSTM-CRF Attention for Academic Knowledge Graph Construction

Academic knowledge graph is an important application of knowledge graph in the vertical field of academia. At present, the construction of the academic knowledge graph is mainly completed by extracting published academic papers, authors, publications, and other information from related databases. However, academic information is not just information of published papers. Scholars&#x2019; academic activities include participation in academic conferences, visiting and making presentation, and so on. However, the above academic information is hidden in natural language texts and cannot be directly stored in academic knowledge graph. This article proposes an approach named construct-SCHOLAT knowledge graph to construct an academic event knowledge graph based on academic social network SCHOLAT. The construction framework mainly consists of two parts: data preprocessing and event extraction. In the data preprocessing, we propose a knowledge graph embedding method to represent scholars&#x2019; academic social feature. In the event extraction, we concatenate the preprocessed scholar vector with academic we-media blog text into the extraction model based on BiLSTM-CRF fused with attention mechanism. The extracted events are added to academic knowledge graph, and a public relationship exists between the event and the scholar. Compared to the previous methods, our framework has an excellent performance after experimental verification. To the best of our knowledge, this is the first study to use the scholar academic social information of the scholar who edited the text as the event extraction input information. In addition, we publish a Chinese event extraction dataset SCHOLAT academic event extraction.https://www.scholat.com/research/opendata The dataset includes academic we-media blog and the social behavior embedding vector of the scholar. All the data in this dataset are derived from the academic social network SCHOLAT.

Unsupervised Adaptive Bipartite Graph Embedding

In traditional graph embedding methods, graph construction is sensitive to high-dimensional data with noise and outliers, making an effective exploration of the neighborhood structure of the data difficult. Besides, with these methods, constructing graphs and reducing dimensions are disconnected and cannot be mutually optimized. To address these problems, we propose an unsupervised dimensionality reduction method based on bipartite graph, named unsupervised adaptive bipartite graph embedding (UABGE). First, the anchors are generated from the raw data by K-means or random sampling. Second, the bipartite graph, which is constructed between the samples and the anchors in the low-dimensional subspace, utilizes the adaptive allocation method to assign neighbors for each sample, so that the local structure of high-dimensional data can be captured effectively. Third, we present an objective function that combines bipartite graph construction and projection matrix learning to achieve mutual optimization between them, which can be solved with an alternating optimization algorithm. Finally, the computational complexity and the convergence of the algorithm are analyzed. Experimental results on synthetic data and publicly available datasets illustrate the effectiveness of the proposed method.

Multiview Deep Graph Infomax to Achieve Unsupervised Graph Embedding.

Unsupervised graph embedding aims to extract highly discriminative node representations that facilitate the subsequent analysis. Converging evidence shows that a multiview graph provides a more comprehensive relationship between nodes than a single-view graph to capture the intrinsic topology. However, little attention has been paid to excavating discriminative representations of each node from multiview heterogeneous networks in an unsupervised manner. To that end, we propose a novel unsupervised multiview graph embedding method, called multiview deep graph infomax (MVDGI). The backbone of our proposed model sought to maximize the mutual information between the view-dependent node representations and the fused unified representation via contrastive learning. Specifically, the MVDGI first uses an encoder to extract view-dependent node representations from each single-view graph. Next, an aggregator is applied to fuse the view-dependent node representations into the view-independent node representations. Finally, a discriminator is adopted to extract highly discriminative representations via contrastive learning. Extensive experiments demonstrate that the MVDGI achieves better performance than the benchmark methods on five real-world datasets, indicating that the obtained node representations by our proposed approach are more discriminative than by its competitors for classification and clustering tasks.

Influence Maximization in Multiagent Systems by a Graph Embedding Method: Dealing With Probabilistically Unstable Links.

This article is concerned with the influence maximization (IM) problem under a network with probabilistically unstable links (PULs) via graph embedding for multiagent systems (MASs). First, two diffusion models, the unstable-link independent cascade (UIC) model and the unstable-link linear threshold (ULT) model, are designed for the IM problem under the network with PULs. Second, the MAS model for the IM problem with PULs is established and a series of interaction rules among agents are built for the MAS model. Third, the similarity of the unstable structure of the nodes is defined and a novel graph embedding method, termed the unstable-similarity2vec (US2vec) approach, is proposed to tackle the IM problem under the network with PULs. According to the embedding results of the US2vec approach, the seed set is figured out by the developed algorithm. Finally, extensive experiments are conducted to: 1) verify the validity of the proposed model and the developed algorithms and 2) illustrate the optimal solution for IM under different scenarios with PULs.

Merit: multi-level graph embedding refinement framework for large-scale graph

The development of the Internet and big data has led to the emergence of graphs as an important data representation structure in various real-world scenarios. However, as data size increases, computational complexity and memory requirements pose significant challenges for graph embedding. To address this challenge, this paper proposes a multilevel embedding refinement framework (MERIT) based on large-scale graphs, using spectral distance-constrained graph coarsening algorithms and an improved graph convolutional neural network model that addresses the over-smoothing problem by incorporating initial values and identity mapping. Experimental results on large-scale datasets demonstrate the effectiveness of MERIT, with an average AUROC score 8% higher than other baseline methods. Moreover, in a node classification task on a large-scale graph with 126,825 nodes and 22,412,658 edges, the framework improves embedding quality while enhancing the runtime by 25 times. The experimental findings highlight the superior efficiency and accuracy of the proposed approach compared to other graph embedding methods.