Graph Embedding Research Articles

Network community detection via neural embeddings

Recent advances in machine learning research have produced powerful neural graph embedding methods, which learn useful, low-dimensional vector representations of network data. These neural methods for graph embedding excel in graph machine learning tasks and are now widely adopted. However, how and why these methods work—particularly how network structure gets encoded in the embedding—remain largely unexplained. Here, we show that node2vec—shallow, linear neural network—encodes communities into separable clusters better than random partitioning down to the information-theoretic detectability limit for the stochastic block models. We show that this is due to the equivalence between the embedding learned by node2vec and the spectral embedding via the eigenvectors of the symmetric normalized Laplacian matrix. Numerical simulations demonstrate that node2vec is capable of learning communities on sparse graphs generated by the stochastic blockmodel, as well as on sparse degree-heterogeneous networks. Our results highlight the features of graph neural networks that enable them to separate communities in the embedding space.

A Strategy for Network multi-layer Information Fusion Based on Multimodel in User Emotional Polarity Analysis

Emotional analysis is an important research direction in natural language processing, which aims to automatically recognize and understand emotions and emotional polarity in texts. The study employed multiple emotional analysis models to analyze emotions in text data. Then the emotional analysis results of different models were integrated to improve accuracy through a hierarchical information fusion strategy. Meanwhile, graph embedding algorithms such as DeepWalk and Node2vec were utilized to process node information in graph data. The embedding representation of graph nodes was utilized for emotional analysis of nodes. In addition, based on models such as graph convolutional neural networks for message passing, contextual information of nodes was obtained to improve emotional analysis performance. These results indicated that the proportion of passive users was higher than that of active users, at 57.14% and 39.68%, respectively. Despite the large number of negative users, the frequency of connections between active users was significantly higher than between negative users. Multiple algorithms were utilized for emotional analysis and prediction. The AUC curve performed well, but the accuracy was only 56.69%, which needs improvement. The study also evaluated the root mean square error of network predictions, with errors typically below 0.2 in large networks, demonstrating relatively accurate prediction results. This guides the further development of emotional analysis technology, promoting the research and improvement of more accurate and reliable emotional analysis methods.

Hierarchically Contrastive Hard Sample Mining for Graph Self-Supervised Pretraining.

Contrastive learning has recently emerged as a powerful technique for graph self-supervised pretraining (GSP). By maximizing the mutual information (MI) between a positive sample pair, the network is forced to extract discriminative information from graphs to generate high-quality sample representations. However, we observe that, in the process of MI maximization (Infomax), the existing contrastive GSP algorithms suffer from at least one of the following problems: 1) treat all samples equally during optimization and 2) fall into a single contrasting pattern within the graph. Consequently, the vast number of well-categorized samples overwhelms the representation learning process, and limited information is accumulated, thus deteriorating the learning capability of the network. To solve these issues, in this article, by fusing the information from different views and conducting hard sample mining in a hierarchically contrastive manner, we propose a novel GSP algorithm called hierarchically contrastive hard sample mining (HCHSM). The hierarchical property of this algorithm is manifested in two aspects. First, according to the results of multilevel MI estimation in different views, the MI-based hard sample selection (MHSS) module keeps filtering the easy nodes and drives the network to focus more on hard nodes. Second, to collect more comprehensive information for hard sample learning, we introduce a hierarchically contrastive scheme to sequentially force the learned node representations to involve multilevel intrinsic graph features. In this way, as the contrastive granularity goes finer, the complementary information from different levels can be uniformly encoded to boost the discrimination of hard samples and enhance the quality of the learned graph embedding. Extensive experiments on seven benchmark datasets indicate that the HCHSM performs better than other competitors on node classification and node clustering tasks. The source code of HCHSM is available at https://github.com/WxTu/HCHSM.

An OD time prediction model based on adaptive graph embedding

The accuracy of origin–destination (OD) time prediction is critical for intelligent transportation systems. To address the spatiotemporal challenges of sparse road network data, this paper introduces an OD time prediction model based on adaptive graph embedding. Firstly, to overcome the issue of limited traffic data, we present a tensor decomposition model specifically designed for estimating missing data. This model focuses on the “time slot-road segment-speed” dimension, dynamically reconstructing absent traffic trajectory data from various perspectives. Secondly, to account for the latent spatial dependencies among road network nodes, we leverage the Node2vec algorithm, which effectively uncovers weak connections in the network. An adaptive Node2vec algorithm, tailored to road network weights, is designed to effectively uncover hidden spatial correlations among adjacent intersections and neighborhoods. Simultaneously, a GRU module is developed to thoroughly explore relevant traffic flow information across distant time periods. Finally, to address the influence of dynamic and static external attributes on predicted travel time, we implement an external environment information fusion module based on the attention mechanism. This module capitalizes on the attention mechanism’s sensitivity to local features, thereby further improving the accuracy of OD time prediction. Experimental results show that the proposed model effectively captures spatiotemporal correlations, delivering superior OD time predictions compared to other baseline methods.

GeoPredict-LLM: Intelligent tunnel advanced geological prediction by reprogramming large language models

With the improvement of multisource information sensing and data acquisition capabilities inside tunnels, the availability of multimodal data in tunnel engineering has significantly increased. However, due to structural differences in multimodal data, traditional intelligent advanced geological prediction models have limited capacity for data fusion. Furthermore, the lack of pre-trained models makes it difficult for neural networks trained from scratch to deeply explore the features of multimodal data. To address these challenges, we utilize the fusion capability of knowledge graph for multimodal data and the pre-trained knowledge of large language models (LLMs) to establish an intelligent advanced geological prediction model (GeoPredict-LLM). First, we develop an advanced geological prediction ontology model, forming a knowledge graph database. Using knowledge graph embeddings, multisource and multimodal data are transformed into low-dimensional vectors with a unified structure. Secondly, pre-trained LLMs, through reprogramming, reconstruct these low-dimensional vectors, imparting linguistic characteristics to the data. This transformation effectively reframes the complex task of advanced geological prediction as a "language-based" problem, enabling the model to approach the task from a linguistic perspective. Moreover, we propose the prompt-as-prefix method, which enables output generation, while freezing the core of the LLM, thereby significantly reduces the number of training parameters. Finally, evaluations show that compared to neural network models without pre-trained models, GeoPredict-LLM significantly improves prediction accuracy. It is worth noting that as long as a knowledge graph database can be established, GeoPredict-LLM can be adapted to multimodal data mining tasks with minimal modifications.

Fuzzy Event Knowledge Graph Embedding Through Event Temporal and Causal Transfer

Fuzzy Event Knowledge Graph Embedding Through Event Temporal and Causal Transfer

Joint embedding in hierarchical distance and semantic representation learning for link prediction

The link prediction task aims to predict missing entities or relations in the knowledge graph and is essential for the downstream application. Existing well-known models deal with this task by mainly focusing on representing knowledge graph triplets in the distance space or semantic space. However, they can not fully capture the information of head and tail entities, nor even make good use of hierarchical level information. Thus, in this paper, we propose a novel knowledge graph embedding model for the link prediction task, namely, HIE, which models each triplet (h, r, t) into distance measurement space and semantic measurement space, simultaneously. Moreover, HIE is introduced into hierarchical-aware space to leverage rich hierarchical information of entities and relations for better representation learning. Specifically, we apply distance transformation operation on the head entity in distance space to obtain the tail entity instead of translation-based or rotation-based approaches. Experimental results of HIE on four real-world datasets show that HIE outperforms several existing state-of-the-art knowledge graph embedding methods on the link prediction task and deals with complex relations accurately.

Physics Embedded Graph Convolution Neural Network for Power Flow Calculation Considering Uncertain Injections and Topology.

Probabilistic analysis tool is important to quantify the impacts of the uncertainties on power system operations. However, the repetitive calculations of power flow are time-consuming. To address this issue, data-driven approaches are proposed but they are not robust to the uncertain injections and varying topology. This article proposes a model-driven graph convolution neural network (MD-GCN) for power flow calculation with high-computational efficiency and good robustness to topology changes. Compared with the basic graph convolution neural network (GCN), the construction of MD-GCN considers the physical connection relationships among different nodes. This is achieved by embedding the linearized power flow model into the layer-wise propagation. Such a structure enhances the interpretability of the network forward propagation. To ensure that enough features are extracted in MD-GCN, a new input feature construction method with multiple neighborhood aggregations and a global pooling layer are developed. This allows us to integrate both global features and neighborhood features, yielding the complete features representation of the system-wide impacts on every single node. Numerical results on the IEEE 30-bus, 57-bus, 118-bus, and 1354-bus systems demonstrate that the proposed method achieves much better performance as compared to other approaches in the presence of uncertain power injections and system topology.

Qudit stabilizer codes, CFTs, and topological surfaces

We study general maps from the space of rational conformal field theories (CFTs) with a fixed chiral algebra and associated Chern-Simons (CS) theories to the space of qudit stabilizer codes with a fixed generalized Pauli group. We consider certain natural constraints on such a map and show that the map can be described as a graph homomorphism from an orbifold graph, which captures the orbifold structure of CFTs, to a code graph, which captures the structure of self-dual stabilizer codes. By studying explicit examples, we show that this graph homomorphism cannot always be a graph embedding. However, we construct a physically motivated map from universal orbifold subgraphs of CFTs to operators in a generalized Pauli group. We show that this map results in a self-dual stabilizer code if and only if the surface operators in the bulk CS theories corresponding to the CFTs in question are self-dual. For CFTs admitting a stabilizer code description, we show that the full Abelianized generalized Pauli group can be obtained from twisted sectors of certain 0-form symmetries of the CFT. Finally, we connect our construction with SymTFTs, and we argue that many equivalences between codes that arise in our setup correspond to equivalence classes of bulk topological surfaces under fusion with invertible surfaces. Published by the American Physical Society 2024

Negative-Free Self-Supervised Gaussian Embedding of Graphs

Graph Contrastive Learning (GCL) has recently emerged as a promising graph self-supervised learning framework for learning discriminative node representations without labels. The widely adopted objective function of GCL benefits from two key properties: alignment and uniformity, which align representations of positive node pairs while uniformly distributing all representations on the hypersphere. The uniformity property plays a critical role in preventing representation collapse and is achieved by pushing apart augmented views of different nodes (negative pairs). As such, existing GCL methods inherently rely on increasing the quantity and quality of negative samples, resulting in heavy computational demands, memory overhead, and potential class collision issues. In this study, we propose a negative-free objective to achieve uniformity, inspired by the fact that points distributed according to a normalized isotropic Gaussian are uniformly spread across the unit hypersphere. Therefore, we can minimize the distance between the distribution of learned representations and the isotropic Gaussian distribution to promote the uniformity of node representations. Our method also distinguishes itself from other approaches by eliminating the need for a parameterized mutual information estimator, an additional projector, asymmetric structures, and, crucially, negative samples. Extensive experiments over seven graph benchmarks demonstrate that our proposal achieves competitive performance with fewer parameters, shorter training times, and lower memory consumption compared to existing GCL methods.

Uncovering Multi-step Attacks with Threat Knowledge Graph Reasoning

The rapid advancement of information technologies has significantly intensified the focus on cyberspace security across various sectors. In this evolving landscape, attackers deploy many of techniques- including exploits, weakness identification, and complex multi-step attacks- to gain unauthorized access to systems. Conversely, defenders harness insights from a variety of sources to pinpoint potential threats. Prominent public cybersecurity databases such as the Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK), Common Attack Pattern Enumeration and Classification (CAPEC), Common Vulnerabilities and Exposures (CVE), Common Weakness Enumeration (CWE), and Common Platform Enumeration (CPE) provide extensive data on security entities and their interrelations, playing a pivotal role in enriching the understanding of cybersecurity challenges and assisting in comprehensive defensive analyses. However, the semantic cross-analysis of these databases, crucial for identifying obscure threat patterns, remains underexploited. In this study, we amalgamate data from these disparate sources into a cohesive threat knowledge graph and introduce a novel knowledge representation learning approach, A4CKGE (ATT&CK-CAPEC-CWE-CVE-CPE Knowledge Graph Embedding). This method utilizes advanced structural and textual analytics to predict interactions among security entities such as products, vulnerabilities, weaknesses, and multi-step attack sequences, employing complex attack templates generated through a Large Language Model (LLM). Our extensive experiments demonstrate that this approach significantly outperforms existing state-of-the-art methods in effectively predicting these relationships. The findings validate the efficacy of our threat knowledge graph in unveiling hidden connections, thereby highlighting its potential to strengthen cybersecurity defenses substantially.

Integrated Knowledge Graph and Drug Molecular Graph Fusion via Adversarial Networks for Drug-Drug Interaction Prediction.

The Co-administration of multiple drugs can enhance the efficacy of disease treatment by reducing drug resistance and side effects. However, it also raises the risk of adverse drug interactions, presenting a challenging problem in healthcare. Various approaches have been developed to predict drug-drug interactions (DDIs) by leveraging both knowledge graphs and drug attribute information. While these methods have shown promise, they often fail to effectively capture correlations between biomedical information in the knowledge graph and drug properties. This work introduces a novel end-to-end DDI predictor framework based on generative adversarial networks. This framework utilizes a message-passing neural network to capture molecular structure information while employing the knowledge-aware graph attention network to capture the representation of drugs in the knowledge graph through considering the importance of different multihop neighbor nodes and relationships. The dual generative adversarial networks employ two generators and two discriminators in knowledge graph embedding and molecular topology embedding for adversarial training to capture the interrelations and complementary knowledge between molecular structure information and semantic information from the knowledge graph. comprehensive experiments have demonstrated that the proposed method outperforms state-of-the-art algorithms in binary classification, with improvements of 1.0% in accuracy, 0.45% in area under the receiver operating characteristic curve (AUC), 0.24% in area under the precision-recall curve (AUPR), and 0.98% in F1 score. Furthermore, for multiclass classification tasks, improvements were observed across various evaluation metrics, including 0.9% in accuracy, 0.25% in macro precision, 0.2% in macro recall, and 0.28% in macro F1. Additionally, ablation studies were conducted to showcase the effectiveness and robustness of our method in DDI prediction tasks.

Development of the Unbiased Disease Network Using Health Insurance Claims Big Data in South Korea

Abstract Background Research on the overall relationship between disease is needed. The disease network model can comprehensively show the complex relationships between diseases and progression patterns. Existing disease network studies were estimated by univariate relative risks. We aim to develop the unbiased disease network. Methods We used NHIS national data of 50 million patients, collected from 2008 to 2022. We used only the primary diagnosis codes (three-digit codes) and defined 603 disease categories based on the special tabulation list for morbidity in the detailed subcategories of ICD-10, prior research, and the frequency of diseases in the dataset. To develop a more unbiased disease network, we first adjusted for age, sex, period, season, and encounter type, then added the confounding disease exposure according to the Directed Acyclic Graph (DAG) and performed Poisson regression to estimate relative risk (RR) applying Inverse Probability Weighting (IPW). We identified statistically significant disease pairs (RR > 1.10, p < 0.001) and repeated the IPW analysis up to the third round because the network would be updated every round. The significant disease pairs identified in the last round were connected to develop the overall network and subnetworks by age and sex. Results We repeated the process up to the third round for all 346,948 disease pairs and fitted 9,024 pairs as significant. We identified general population disease network and subnetwork per age and sex subgroups. Conclusions These disease networks represented the overall progression relationship between diagnosis codes. Key messages • We developed the unbiased disease network that shows progression patterns by adjusting for confounders and using IPW. • The disease network can be used to extract features from various diagnosis codes in EHR/Claims data, analyze causal effects, and predict disease or healthcare usage using RR or graph embedding.

Face‐simple minimal quadrangulations of surfaces

AbstractFor each surface besides the sphere, projective plane, and Klein bottle, we construct a face‐simple minimal quadrangulation, that is, a simple quadrangulation on the fewest number of vertices possible, whose dual is also a simple graph. Our result answers a question of Liu, Ellingham, and Ye while providing a simpler proof of their main result. The inductive construction is based on an earlier idea for finding near‐quadrangular embeddings of the complete graphs using the diamond sum operation.

GEMimp: An Accurate and Robust Imputation Method for Microbiome Data Using Graph Embedding Neural Network

Microbiome research has increasingly underscored the profound link between microbial compositions and human health, with numerous studies establishing a strong correlation between microbiome characteristics and various diseases. However, the analysis of microbiome data is frequently compromised by inherent sparsity issues, characterized by a substantial presence of observed zeros. These zeros not only skew the abundance distribution of microbial species but also undermine the reliability of scientific conclusions drawn from such data. Addressing this challenge, we introduce GEMimp, an innovative imputation method designed to infuse robustness into microbiome data analysis. GEMimp leverages the node2vec algorithm, which incorporates both Breadth-First Search (BFS) and Depth-First Search (DFS) strategies in its random walks sampling process. This approach enables GEMimp to learn nuanced, low-dimensional representations of each taxonomic unit, facilitating the reconstruction of their similarity networks with unprecedented accuracy.Our comparative analysis pits GEMimp against state-of-the-art imputation methods including SAVER, MAGIC and mbImpute. The results unequivocally demonstrate that GEMimp outperforms its counterparts by achieving the highest Pearson correlation coefficient when compared to the original raw dataset. Furthermore, GEMimp shows notable proficiency in identifying significant taxa, enhancing the detection of disease-related taxa and effectively mitigating the impact of sparsity on both simulated and real-world datasets, such as those pertaining to Type 2 Diabetes (T2D) and Colorectal Cancer (CRC). These findings collectively highlight the strong effectiveness of GEMimp, allowing for better analysis on microbial data. With alleviation of sparsity issues, it could be greatly facilitated in downstream analyses and even in the field of microbiology.

Dual De-confounded Causal Intervention method for knowledge graph error detection

Due to the Knowledge Graph (KG) construction process, erroneous triples are virtually inevitable to be introduced into real-world KGs. Since these errors hinder the expressiveness and applicability of KGs, the development of knowledge graph error detection (KGED) methods is necessary. Despite the overall effectiveness of current KGED methods, their capacity to identify challenging errors is limited. In this work, we conduct empirical studies and find that previous works introduce structural and semantic bias, impeding the identification of erroneous triples, especially in challenging cases. To address this issue, we design a causal graph for the KGED task and propose a Dual De-confounded Causal Intervention (DuDCI) method for debiasing. Firstly, DuDCI utilizes the neighborhood and textual descriptions of triples to calculate their graph and text embeddings. Next, a Causal De-confounded Module is constructed to mitigate the impact of shortcuts caused by the bias through the front-door adjustment. Furthermore, we introduce Disentanglement Constraints to disentangle the information expressed by each embedding, thereby facilitating further bias mitigation. Experimental results on three widely used KGED datasets validate the effectiveness of DuDCI and demonstrate that DuDCI outperforms current KGED methods, with an improvement of at least 2.2%, especially in more challenging noise scenarios.

Dual semi-supervised hypergraph regular multi-view NMF with anchor graph embedding

Graph regularized nonnegative matrix factorization (GNMF) has been widely used in multi-view clustering tasks due to its good clustering properties. However, it uses a simple graph to describe the complex data relationship of multiple views and tries to obtain a consistent low-dimensional representation, which undoubtedly brings challenges to its clustering performance. In addition, clustering a large amount of high-dimensional data from multiple views undoubtedly faces a huge computational burden. In order to effectively improve the performance and efficiency of GNMF based multi-view clustering algorithm, this paper proposes a dual semi-supervised hypergraph regular multi-view clustering method with anchor graph embedding (DSSHMNMFAE). Specifically, DSSHMNMFAE develops a new anchor selection method to generate anchors and the anchor bipartite graph is constructed to embed the matrix factorization process. DSSHMNMFAE constructs a hypergraph to effectively learn the high-order relationship between data from multiple views. In order to perform semi-supervised learning more efficiently, DSSHMNMFAE integrates pairwise constraint information and label constraint information into the clustering process as dual label information. In addition, DSSHMNMFAE considers the learning of both consistency information and complementarity information, and adopts adaptive measures to distinguish the contributions of different views. We use the alternating iterative algorithm to optimize the objective function of DSSHMNMFAE. The experimental results on eight real datasets show that the performance of DSSHMNMFAE is comparable to other algorithms.

From communities to interpretable network and word embedding: an unified approach

Abstract Modelling information from complex systems such as humans social interaction or words co-occurrences in our languages can help to understand how these systems are organized and function. Such systems can be modelled by networks, and network theory provides a useful set of methods to analyze them. Among these methods, graph embedding is a powerful tool to summarize the interactions and topology of a network in a vectorized feature space. When used in input of machine learning algorithms, embedding vectors help with common graph problems such as link prediction, graph matching, etc In Natural Language Processing (NLP), such a vectorization process is also employed. Word embedding has the goal of representing the sense of words, extracting it from large text corpora. Despite differences in the structure of information in input of embedding algorithms, many graph embedding approaches are adapted and inspired from methods in NLP. Limits of these methods are observed in both domains. Most of these methods require long and resource greedy training. Another downside to most methods is that they are black-box, from which understanding how the information is structured is rather complex. Interpretability of a model allows understanding how the vector space is structured without the need for external information, and thus can be audited more easily. With both these limitations in mind, we propose a novel framework to efficiently embed network vertices in an interpretable vector space. Our Lower Dimension Bipartite Framework (LDBGF) leverages the bipartite projection of a network using cliques to reduce dimensionality. Along with LDBGF, we introduce two implementations of this framework that rely on communities instead of cliques: SINr-NR and SINr-MF. We show that SINr-MF can perform well on classical graphs and SINr-NR can produce high-quality graph and word embeddings that are interpretable and stable across runs.

Multi-level discriminator based contrastive learning for multiplex networks

Graph embedding is a technique for obtaining low-dimensional representations of nodes across diverse networks, which may then be used for various downstream tasks and applications. When it applies to heterogeneous networks, it is hard to handle heterogeneous networks because they usually contain different types of nodes and edges with more semantic and structural information. Recently, contrastive learning has developed as the preferred strategy for dealing with unsupervised heterogeneous graph embedding to reduce the cost of human label annotation. However, most multi-view contrastive learning approaches calculate the model’s loss only based on the mutual dependence between the node representation and graph representation. These approaches ignore that both node attributes and node clustering contain discriminative content. To solve this issue, we propose a model called Multi-Level Discriminator-based Contrastive Learning for Multiplex Networks (MLDCL). This model adopts a multi-level multi-discriminator-based approach that can simultaneously learn the global-level structural information, node-level attribute information, and local-level clustering information. Moreover, an augmentation strategy in the contrast learning process from the spectral domain is proposed to improve the representation and discriminative ability of MLDCL. Numerous tests with node clustering and classification tasks on widely used datasets demonstrate the efficacy of the proposed approach.

Maximizing Influence in Social Networks Using Combined Local Features and Deep Learning-Based Node Embedding.

The influence maximization problem has several issues, including low infection rates and high time complexity. Many proposed methods are not suitable for large-scale networks due to their time complexity or free parameter usage. To address these challenges, this article proposes a local heuristic called Embedding Technique for Influence Maximization (ETIM) that uses shell decomposition, graph embedding, and reduction, as well as combined local structural features. The algorithm selects candidate nodes based on their connections among network shells and topological features, reducing the search space and computational overhead. It uses a deep learning-based node embedding technique to create a multidimensional vector of candidate nodes and calculates the dependency on spreading for each node based on local topological features. Finally, influential nodes are identified using the results of the previous phases and newly defined local features. The proposed algorithm is evaluated using the independent cascade model, showing its competitiveness and ability to achieve the best performance in terms of solution quality. Compared with the collective influence global algorithm, ETIM is significantly faster and improves the infection rate by an average of 12%.