Low-dimensional Representations Of Nodes Research Articles

Multi-level discriminator based contrastive learning for multiplex networks

Graph embedding is a technique for obtaining low-dimensional representations of nodes across diverse networks, which may then be used for various downstream tasks and applications. When it applies to heterogeneous networks, it is hard to handle heterogeneous networks because they usually contain different types of nodes and edges with more semantic and structural information. Recently, contrastive learning has developed as the preferred strategy for dealing with unsupervised heterogeneous graph embedding to reduce the cost of human label annotation. However, most multi-view contrastive learning approaches calculate the model’s loss only based on the mutual dependence between the node representation and graph representation. These approaches ignore that both node attributes and node clustering contain discriminative content. To solve this issue, we propose a model called Multi-Level Discriminator-based Contrastive Learning for Multiplex Networks (MLDCL). This model adopts a multi-level multi-discriminator-based approach that can simultaneously learn the global-level structural information, node-level attribute information, and local-level clustering information. Moreover, an augmentation strategy in the contrast learning process from the spectral domain is proposed to improve the representation and discriminative ability of MLDCL. Numerous tests with node clustering and classification tasks on widely used datasets demonstrate the efficacy of the proposed approach.

ClusterLP: A novel Cluster-aware Link Prediction model in undirected and directed graphs

Link prediction models endeavor to understand the distribution of links within graphs and forecast the presence of potential links. With the advancements in deep learning, prevailing methods typically strive to acquire low-dimensional representations of nodes in networks, aiming to capture and retain the structure and inherent characteristics of networks. However, the majority of these methods primarily focus on preserving the microscopic structure, such as the first- and second-order proximities of nodes, while largely disregarding the mesoscopic cluster structure, which stands out as one of the network's most prominent features. Following the homophily principle, nodes within the same cluster exhibit greater similarity to each other compared to those from different clusters, suggesting that they should possess analogous vertex representations and higher probabilities of linkage. In this study, we develop a straightforward yet efficient Cluster-aware Link Prediction framework (ClusterLP), with the objective of directly leveraging cluster structures to predict links among nodes with maximum accuracy in both undirected and directed graphs. Specifically, we posit that establishing links between nodes with similar representation vectors and cluster tendencies is more feasible in undirected graphs, whereas nodes in directed graphs are inclined to point towards nodes with akin representation vectors and greater influence. We tailor the implementation of ClusterLP for undirected and directed graphs, respectively, and experimental findings using multiple real-world networks demonstrate the high competitiveness of our models in the realm of link prediction tasks. The code utilized in our implementation is accessible at https://github.com/ZINUX1998/ClusterLP.

IMNE: Maximizing influence through deep learning-based node embedding in social network

Influence Maximization (IM) is a critical problem in social network analysis and marketing. It involves identifying a subset of nodes in a social network whose activation or influence can lead to the maximal spread of information, ideas, or behaviors within the network. Although many approaches have been developed in the literature to deal with this problem, most of these approaches are ineffective in dealing with large-scale social networks due to free parameters and computational complexity. Embeddings are used to learn low-dimensional representations of nodes in a social network. These embeddings capture the structural and semantic information of nodes and their relationships within the network. By training deep learning models on graph-structured data, node embeddings can capture complex patterns and dependencies in social networks, enabling more effective downstream tasks such as IM. Accordingly, this paper proposes an efficient algorithm to address the IM problem in social networks using deep learning-based Node Embedding (IMNE), which includes shell decomposition, graph/node embedding, and search space reduction as well as the use of local structural features. Our approach combines the power of deep learning for representation learning with the rich structural information present in social networks to address the challenge of IM in complex and dynamic social networks. IMNE uses the Independent Cascade (IC) information diffusion model to determine the labels needed to train the model by calculating the influence of nodes. Experimental results on several real-world networks considering different performance metrics show that IMNE performs better compared to existing baseline and state-of-the-art methods.

Status-Aware Signed Heterogeneous Network Embedding With Graph Neural Networks.

Many real-world applications are inherently modeled as signed heterogeneous networks or graphs with positive and negative links. Signed graph embedding embeds rich structural and semantic information of a signed graph into low-dimensional node representations. Existing methods usually exploit social structural balance theory to capture the semantics of the complex structure in a signed graph. These methods either omit the node features or may discard the direction information of the links. To address these issues, we propose a new framework, called a status-aware graph neural network (S-GNN), to boost the representation learning performance. S-GNN is equipped with a loss function designed based on status theory, a social-psychological theory specifically developed for directed signed graphs. Extensive experimental results on benchmarking datasets verified that S-GNN can distill comprehensive information ingrained in a signed graph in the embedding space. Specifically, S-GNN achieves state-of-the-art accuracy, robustness, and scalability: it speeds up the processing time of link sign prediction by up to 6.5 × and increases accuracy by up to 18.8% as compared with the alternatives. We also show that S-GNN can obtain effective status scores of nodes for link sign prediction and node ranking tasks, both of which yield state-of-the-art performance.

HeGAE-AC: Heterogeneous graph auto-encoder for attribute completion

Heterogeneous Graphs (HGs) are widely used to model complex real-world networks that involve multiple types of nodes and relations. Many heterogeneous graph neural networks (HGNNs) based methods have been developed to learn low-dimensional representations for nodes in HGs with the assumption of complete attributes for all nodes. However, in many real applications, the attributes of certain nodes may be completely missing, leading to attribute-missing issues. Existing methods for attribute completion in HGs separate topological embedding learning from attribute completion, which can result in sub-optimal solutions. Moreover, the information of higher-order connected nodes is not fully exploited in the process of attribute completion. To address the issues mentioned above, we integrate topological embedding learning, attribute completion, and heterogeneous graph representation learning into an end-to-end framework called HeGAE-AC. Firstly, we utilize a heterogeneous graph auto-encoder (HeGAE) to incorporate the graph topology structure and attribute information of higher-order connected nodes into low-dimensional embeddings. Then, HeGAE decodes embeddings to generate attributes of all nodes. Finally, an HGNN model is employed to obtain node representations for downstream tasks using the completed HGs. We conduct extensive experiments on four real-world networks, which demonstrate that our approach achieves significant improvements over state-of-the-art methods.

Improved graph representation learning based on neighborhood aggregation and interaction fusion

The adaptive fusion module with an attention mechanism functions by employing a dual-channel graph convolutional network to aggregate neighborhood information. The resulting embeddings are then utilized to calculate interaction terms, thereby incorporating additional information. To enhance the relevance of fusion information, an adaptive fusion module with an attention mechanism is constructed. This module selectively combines the neighborhood aggregation and interaction terms, prioritizing the most pertinent information. Through this adaptive fusion process, the algorithm effectively captures both neighborhood features and other nonlinear information, leading to improved overall performance. Neighborhood Aggregation Interaction Graph Convolutional Network Adaptive Fusion (NAIGCNAF) is a graph representation learning algorithm designed to obtain low-dimensional node representations while preserving graph properties. It addresses the limitations of existing algorithms, which tend to focus solely on aggregating neighborhood features and overlook other nonlinear information. NAIGCNAF utilizes a dual-channel graph convolutional network for neighborhood aggregation and calculates interaction terms based on the resulting embeddings. Additionally, it incorporates an adaptive fusion module with an attention mechanism to enhance the relevance of fusion information. Extensive evaluations on three citation datasets demonstrate that NAIGCNAF outperforms other algorithms such as GCN, Neighborhood Aggregation, and AIR-GCN. NAIGCNAF achieves notable improvements in classification accuracy, ranging from 1.0 to 1.6 percentage points on the Cora dataset, 1.1 to 2.4 percentage points on the Citeseer dataset, and 0.3 to 0.9 percentage points on the Pubmed dataset. Moreover, in visualization tasks, NAIGCNAF exhibits clearer boundaries and stronger aggregation within clusters, enhancing its effectiveness. Additionally, the algorithm showcases faster convergence rates and smoother accuracy curves, further emphasizing its ability to improve benchmark algorithm performance.

SHEEP, a Signed Hamiltonian Eigenvector Embedding for Proximity

Signed network embedding methods allow for a low-dimensional representation of nodes and primarily focus on partitioning the graph into clusters, hence losing information on continuous node attributes. Here, we introduce a spectral embedding algorithm for understanding proximal relationships between nodes in signed graphs, where edges can take either positive or negative weights. Inspired by a physical model, we construct our embedding as the minimum energy configuration of a Hamiltonian dependent on the distance between nodes and locate the optimal embedding dimension. We show through a series of experiments on synthetic and empirical networks, that our method (SHEEP) can recover continuous node attributes showcasing its main advantages: re-configurability into a computationally efficient eigenvector problem, retrieval of ground state energy which can be used as a statistical test for the presence of strong balance, and measure of node extremism, computed as the distance to the origin in the optimal embedding.

Inductive Link Prediction via Interactive Learning Across Relations in Multiplex Networks

Network embedding is an important class of link prediction methods, which can use the distance between learned low-dimensional node representations to characterize the similarity between nodes. Traditional network embedding methods focus on single-layer networks, while in reality, a large part of complex networks are not isolated, but interdependent and interrelated, forming multiplex complex networks. Also, how to effectively exploit layer correlations in multiplex networks to learn more robust and valuable representations, to improve link prediction performance, has been a hot research topic in the field of complex network analysis. However, previous studies mainly focus on inferring intralinks in each layer of complex networks or anchor links among layers. Another issue that has not been discussed is how to predict potential links or reconstruct the network in unobserved relations based on existing multiplex networks. To this issue, we define a novel inductive link prediction problem in multiplex networks, in which most existing multichannel network embedding methods fail to solve. This is either because they only emphasize the specific structure information of an individual layer or only capture the common information for all layers. To effectively address this problem, we propose a novel embedding method termed interactive learning across relations (ILAR), to capture and fully exploit the multiple relations and complex layer correlations in multiplex networks. We leverage two convolutional modules and ILAR to capture the sufficient complementary and correlations in multiplex networks. Moreover, during interactive learning, a disparity constraint is introduced, which enforces the features encoded from two convolutional modules to be different and prevents information redundancy. Finally, the extensive experiments in several real-world datasets show that our model can significantly outperform the existing state-of-the-art network embedding methods on the novel link prediction problem in multiplex networks.

Heterogeneous Evolution Network Embedding with Temporal Extension for Intelligent Tutoring Systems

Graph embedding (GE) aims to acquire low-dimensional node representations while maintaining the graph’s structural and semantic attributes. Intelligent tutoring systems (ITS) signify a noteworthy achievement in the fusion of AI and education. Utilizing GE to model ITS can elevate their performance in predictive and annotation tasks. Current GE techniques, whether applied to heterogeneous or dynamic graphs, struggle to efficiently model ITS data. The GEs within ITS should retain their semidynamic, independent, and smooth characteristics. This article introduces a heterogeneous evolution network (HEN) for illustrating entities and relations within an ITS. Additionally, we introduce a temporal extension graph neural network (TEGNN) to model both evolving and static nodes within the HEN. In the TEGNN framework, dynamic nodes are initially improved over time through temporal extension (TE), providing an accurate depiction of each learner’s implicit state at each time step. Subsequently, we propose a stochastic temporal pooling (STP) strategy to estimate the embedding sets of all evolving nodes. This effectively enhances model efficiency and usability. Following this, a heterogeneous aggregation network is devised to proficiently extract heterogeneous features from the HEN. This network employs both node-level and relation-level attention mechanisms to craft aggregated node features. To emphasize the superiority of TEGNN, we perform experiments on several real ITS datasets and show that our method significantly outperforms the state-of-the-art approaches. The experiments validate that TE serves as an efficient framework for modeling temporal information in GE, and STP not only accelerates the training process but also enhances the resultant accuracy.

Graphfool: Targeted Label Adversarial Attack on Graph Embedding

Graph embedding learns low-dimensional representations for nodes or edges on the graph, which is widely applied in many real-world applications. Excessive graph mining promotes the research of attack methods on graph embedding. Most attack methods generate perturbations that maximize the deviation of the prediction confidence. They are difficult to accurately misclassify the instances into the target label, and the nonminimized perturbations are more easily detected by defense methods. To address these problems, we propose Graphfool, a novel targeted label adversarial attack on graph embedding. It can generate adversarial graphs to attack graph embedding methods via classification boundary and gradient information in the target graph embedding method. Graphfool first estimates the classification boundaries of different categories. Then, it calculates the minimum perturbation matrix to misclassify the attacked node according to the target classification boundary. Finally, the adjacency matrix is modified according to the maximum absolute value of the perturbation matrix. Extensive experiments demonstrate that Graphfool achieves the state-of-the-art attack performance with minimum perturbations. Besides, the possible defense experiments further prove that the perturbations generated by Graphfool are more imperceptible.

An effective multi-task learning framework for drug repurposing based on graph representation learning.

Drug repurposing, which typically applies the procedure of drug-disease associations (DDAs) prediction, is a feasible solution to drug discovery. Compared with traditional methods, drug repurposing can reduce the cost and time for drug development and advance the success rate of drug discovery. Although many methods for drug repurposing have been proposed and the obtained results are relatively acceptable, there is still some room for improving the predictive performance, since those methods fail to consider fully the issue of sparseness in known drug-disease associations. In this paper, we propose a novel multi-task learning framework based on graph representation learning to identify DDAs for drug repurposing. In our proposed framework, a heterogeneous information network is first constructed by combining multiple biological datasets. Then, a module consisting of multiple layers of graph convolutional networks is utilized to learn low-dimensional representations of nodes in the constructed heterogeneous information network. Finally, two types of auxiliary tasks are designed to help to train the target task of DDAs prediction in the multi-task learning framework. Comprehensive experiments are conducted on real data and the results demonstrate the effectiveness of the proposed method for drug repurposing.

IHerd: an integrative hierarchical graph representation learning framework to quantify network changes and prioritize risk genes in disease.

Different genes form complex networks within cells to carry out critical cellular functions, while network alterations in this process can potentially introduce downstream transcriptome perturbations and phenotypic variations. Therefore, developing efficient and interpretable methods to quantify network changes and pinpoint driver genes across conditions is crucial. We propose a hierarchical graph representation learning method, called iHerd. Given a set of networks, iHerd first hierarchically generates a series of coarsened sub-graphs in a data-driven manner, representing network modules at different resolutions (e.g., the level of signaling pathways). Then, it sequentially learns low-dimensional node representations at all hierarchical levels via efficient graph embedding. Lastly, iHerd projects separate gene embeddings onto the same latent space in its graph alignment module to calculate a rewiring index for driver gene prioritization. To demonstrate its effectiveness, we applied iHerd on a tumor-to-normal GRN rewiring analysis and cell-type-specific GCN analysis using single-cell multiome data of the brain. We showed that iHerd can effectively pinpoint novel and well-known risk genes in different diseases. Distinct from existing models, iHerd's graph coarsening for hierarchical learning allows us to successfully classify network driver genes into early and late divergent genes (EDGs and LDGs), emphasizing genes with extensive network changes across and within signaling pathway levels. This unique approach for driver gene classification can provide us with deeper molecular insights. The code is freely available at https://github.com/aicb-ZhangLabs/iHerd. All other relevant data are within the manuscript and supporting information files.

Amer: A New Attribute-Missing Network Embedding Approach.

Network embedding which aims to learn a low dimensional representation of nodes is a powerful technique for network analysis. While network embedding for networks with complete attributes has been widely investigated, in many real-world applications the attributes of partial nodes are unobserved (i.e., missing) due to privacy concern or resource limit. Very recently, several network embedding methods have been proposed for attribute-missing networks. They first complete the missing attributes and then use the complemented network to learn network embedding. The parameters of these two processes cannot be adjusted by each other, resulting in compromised results. To address this problem, we propose a unified model in which the process of completing missing attributes and the process of learning embedding are not separated but closely intertwined. Being specific, completing missing attributes is under the guidance of learning network representation via mutual information maximization, and the complemented attributes directly enter network representation module which will generate further feedback for completing missing attributes. We further impose attribute-structure relationship constraint for completing missing attributes by designing a new generative adversarial networks (GANs) model. To the best of our knowledge, this is the first unified model for attribute-missing network embedding. Empirical results on real-world datasets show the superiority of our new method over other state-of-the-art methods on four network analysis tasks, including node classification, node clustering, link prediction, and network visualization.

Attribute Augmented Network Embedding Based on Generative Adversarial Nets.

Network embedding is to learn low-dimensional representations of nodes while preserving necessary information for network analysis tasks. Though representations preserving both structure and attribute features have achieved in many real-world applications, learning these representations for networks with attribute information is difficult due to the heterogeneity between structure and attribute information. Many existing methods have been proposed to preserve explicit proximities between nodes, with optimization limited to node pairs with large structure and attribute proximities, which may lead to overfitting. To address the above problems, we adopt an attribute augmented network to represent attribute and structure information in a unified framework. Specifically, we study the problem of attribute augmented network embedding that exploits the strength of generative adversarial nets (ANGANs) in capturing the latent distribution of data to learn robust and informative representations of nodes. The ANGAN method obtains the low-dimensional representations of nodes through adversarial learning between the generative and discriminative models. The generative model approximates the underlying connectivity and attributes distributions of nodes by using the distributions generated from the learned representations. It is implemented by utilizing the properties of the attribute augmented network to improve the traditional Skip-gram model. The discriminative model is designed as a binary classifier to distinguish the truly connected node pairs from the generated ones. The pre-training algorithm and the teacher forcing approach are adopted to improve training efficiency and stability. Empirical results show that ANGAN generally outperforms state-of-the-art methods in various real-world applications, which demonstrates the effectiveness and generality of our method.

Relation-Aware Weighted Embedding for Heterogeneous Graphs

Heterogeneous graph embedding, aiming to learn the low-dimensional representations of nodes, is effective in many tasks, such as link prediction, node classification, and community detection. Most existing graph embedding methods conducted on heterogeneous graphs treat the heterogeneous neighbours equally. Although it is possible to get node weights through attention mechanisms mainly developed using expensive recursive message-passing, they are difficult to deal with large-scale networks. In this paper, we propose R-WHGE, a relation-aware weighted embedding model for heterogeneous graphs, to resolve this issue. R-WHGE comprehensively considers structural information, semantic information, meta-paths of nodes and meta-path-based node weights to learn effective node embeddings. More specifically, we first extract the feature importance of each node and then take the nodes’ importance as node weights. A weighted random walks-based embedding learning model is proposed to generate the initial weighted node embeddings according to each meta-path. Finally, we feed these embeddings to a relation-aware heterogeneous graph neural network to generate compact embeddings of nodes, which captures relation-aware characteristics. Extensive experiments on real-world datasets demonstrate that our model is competitive against various state-of-the-art methods.

Multi-View Graph Neural Architecture Search for Biomedical Entity and Relation Extraction.

Recently, graph neural architecture search (GNAS) frameworks have been successfully used to automatically design the optimal neural architectures for many problems such as node classification and graph classification. In the existing GNAS frameworks, the designed graph neural network (GNN) architectures learn the representation of homogenous graphs with one type of relationship connecting two nodes. However, multi-view graphs, where each view represents a type of relationship among nodes, are ubiquitous in the real world. The traditional GNAS frameworks learn the graph representation without considering the interactions between nodes and multiple relationships, so they fail to solve multi-view graph-based problems, such as multi-view graphs modelling the biomedical entity and relation extraction tasks. In this paper, we propose MVGNAS, a multi-view graph neural network automatic modelling framework for biomedical entity and relation extraction, to resolve this challenge. In MVGNAS, we propose an automatic multi-view representation learning to learn low-dimensional representations of nodes that capture multiple relationships in a multi-view graph, representing the first research work in literature to solve the problem of multi-view graph representation learning architecture search for biomedical entity and relation extraction tasks. The experimental results demonstrate that MVGNAS can achieve the best performance in biomedical entity and relation extraction tasks against the state-of-the-art baseline methods.

HetIG-PreDiG: A Heterogeneous Integrated Graph Model for Predicting Human Disease Genes based on gene expression.

Graph analytical approaches permit identifying novel genes involved in complex diseases, but are limited by (i) inferring structural network similarity of connected gene nodes, ignoring potentially relevant unconnected nodes; (ii) using homogeneous graphs, missing gene-disease associations' complexity; (iii) relying on disease/gene-phenotype associations' similarities, involving highly incomplete data; (iv) using binary classification, with gene-disease edges as positive training samples, and non-associated gene and disease nodes as negative samples that may include currently unknown disease genes; or (v) reporting predicted novel associations without systematically evaluating their accuracy. Addressing these limitations, we develop the Heterogeneous Integrated Graph for Predicting Disease Genes (HetIG-PreDiG) model that includes gene-gene, gene-disease, and gene-tissue associations. We predict novel disease genes using low-dimensional representation of nodes accounting for network structure, and extending beyond network structure using the developed Gene-Disease Prioritization Score (GDPS) reflecting the degree of gene-disease association via gene co-expression data. For negative training samples, we select non-associated gene and disease nodes with lower GDPS that are less likely to be affiliated. We evaluate the developed model's success in predicting novel disease genes by analyzing the prediction probabilities of gene-disease associations. HetIG-PreDiG successfully predicts (Micro-F1 = 0.95) gene-disease associations, outperforming baseline models, and is validated using published literature, thus advancing our understanding of complex genetic diseases.

Enhanced Signed Graph Neural Network with Node Polarity.

Signed graph neural networks learn low-dimensional representations for nodes in signed networks with positive and negative links, which helps with many downstream tasks like link prediction. However, most existing signed graph neural networks ignore individual characteristics of nodes and thus limit the ability to learn the underlying structure of real signed graphs. To address this limitation, a deep graph neural network framework SiNP to learn Signed network embedding with Node Polarity is proposed. To be more explicit, a node-signed property metric mechanism is developed to encode the individual characteristics of the nodes. In addition, a graph convolution layer is added so that both positive and negative information from neighboring nodes can be combined. The final embedding of nodes is produced by concatenating the outcomes of these two portions. Finally, extensive experiments have been conducted on four significant real-world signed network datasets to demonstrate the efficiency and superiority of the proposed method in comparison to the state-of-the-art.

MUSE: Multi-faceted attention for signed network embedding

Signed network embedding is an approach to learning low-dimensional representations of nodes in signed networks with both positive and negative links, which facilitates downstream tasks such as link prediction with general data mining frameworks. Due to the distinct properties and significant added value of negative links, existing signed network embedding methods usually design dedicated methods based on social theories such as balance theory and status theory. However, existing signed network embedding methods ignore the characteristics of multiple facets of each node and mix them up in one single representation, which limits the ability to capture the fine-grained attentions between node pairs. In this paper, we propose MUSE, a MUlti-faceted attention-based Signed network Embedding framework to tackle this problem. Specifically, a joint intra- and inter-facet attention mechanism is introduced to aggregate fine-grained information from neighbor nodes. Moreover, balance theory is also utilized to guide information aggregation from multi-order balanced and unbalanced neighbors. Experimental results on four real-world signed network datasets demonstrate the effectiveness of our proposed framework.

Personalised meta-path generation for heterogeneous graph neural networks

Recently, increasing attention has been paid to heterogeneous graph representation learning (HGRL), which aims to embed rich structural and semantic information in heterogeneous information networks (HINs) into low-dimensional node representations. To date, most HGRL models rely on hand-crafted meta-paths. However, the dependency on manually-defined meta-paths requires domain knowledge, which is difficult to obtain for complex HINs. More importantly, the pre-defined or generated meta-paths of all existing HGRL methods attached to each node type or node pair cannot be personalised to each individual node. To fully unleash the power of HGRL, we present a novel framework, Personalised Meta-path based Heterogeneous Graph Neural Networks (PM-HGNN), to jointly generate meta-paths that are personalised for each individual node in a HIN and learn node representations for the target downstream task like node classification. Precisely, PM-HGNN treats the meta-path generation as a Markov Decision Process and utilises a policy network to adaptively generate a meta-path for each individual node and simultaneously learn effective node representations. The policy network is trained with deep reinforcement learning by exploiting the performance improvement on a downstream task. We further propose an extension, PM-HGNN++, to better encode relational structure and accelerate the training during the meta-path generation. Experimental results reveal that both PM-HGNN and PM-HGNN++ can significantly and consistently outperform 16 competing baselines and state-of-the-art methods in various settings of node classification. Qualitative analysis also shows that PM-HGNN++ can identify meaningful meta-paths overlooked by human knowledge.