Low-dimensional Representations Of Nodes Research Articles

Node Pair Information Preserving Network Embedding Based on Adversarial Networks.

Network embedding aims to learn the low-dimensional node representations for networks, which has attracted an increasing amount of attention in recent years. Most existing efforts in this field attempt to embed the network based on node similarity, which generally relies on edge existence statistics of the network. Instead of relying on the global edge existence statistics for every node pair, in this article, we utilize the information between a pair of nodes in a local way and propose a model, called node pair information preserving network embedding (NINE), based on adversarial networks. The main idea lies in preserving the node pair information (NI) by means of adversarial networks. The architecture of the proposed NINE model consists of three main components, namely: 1) NI embedder; 2) NI generator; and 3) NI discriminator. In the NI embedder, to avoid the complicated similarity calculation for a pair of nodes, the original NI vector calculated from the direct neighbor information of the two nodes is adopted as features, and the edge existence information is taken as labels to learn the embedded NI vector in a supervised learning manner. The second component is the NI generator, which takes the original node representation vectors of a node pair as input and outputs the generated NI vector. In order to make the generated NI vector follow the same distribution of the corresponding embedded NI vector, the generative adversarial network (GAN) is adopted, resulting in the third component, called the NI discriminator. Extensive experiments are conducted on seven real-world datasets in three downstream tasks, namely: 1) network reconstruction; 2) link prediction; and 3) node classification. Comparison results with seven state-of-the-art models demonstrate the effectiveness, efficiency, and rationality of our model.

A Novel Global Prototype-Based Node Embedding Technique

Node embedding refers to learning or generating low-dimensional representations for nodes in a given graph. In the era of big data and large graphs, there has been a growing interest in node embedding across a wide range of applications, ranging from social media to healthcare. Numerous research efforts have been invested in searching for node embeddings that maximally preserve the associated graph properties. However, each embedding technique has its own relative disadvantages. This paper presents a method for generating deep neural node embeddings that encode dissimilarity scores between pairs of nodes with the help of prototype nodes spread throughout the target graph. The technique we develop is adaptable to various notions of dissimilarity and yields efficient embeddings capable of estimating the dissimilarity between any two pairs of nodes in a graph. We compare our technique against some state-of-the-art similar embedding techniques and demonstrate superior results in a number of experiments using several benchmark datasets.

A Normalizing Flow-Based Co-Embedding Model for Attributed Networks

Network embedding is a technique that aims at inferring the low-dimensional representations of nodes in a semantic space. In this article, we study the problem of inferring the low-dimensional representations of both nodes and attributes for attributed networks in the same semantic space such that the affinity between a node and an attribute can be effectively measured. Intuitively, this problem can be addressed by simply utilizing existing variational auto-encoder (VAE) based network embedding algorithms. However, the variational posterior distribution in previous VAE based network embedding algorithms is often assumed and restricted to be a mean-field Gaussian distribution or other simple distribution families, which results in poor inference of the embeddings. To alleviate the above defect, we propose a novel VAE-based co-embedding method for attributed network, F-CAN, where posterior distributions are flexible, complex, and scalable distributions constructed through the normalizing flow. We evaluate our proposed models on a number of network tasks with several benchmark datasets. Experimental results demonstrate that there are clear improvements in the qualities of embeddings generated by our model to the state-of-the-art attributed network embedding methods.

Heterogeneous graph embedding by aggregating meta-path and meta-structure through attention mechanism

Heterogeneous information networks embedding, which is a promising technique to learn low-dimensional representations for nodes with different types, has obtained very high interests recently. Plenty of graph neural networks models have been proposed for heterogeneous graph embedding. However, there are two limitations in existing models: (1) despite the complex structures of heterogeneous nodes, almost all these models are mainly based on meta-paths; (2) the different importance of different structures are neglected in most models. To address these problems, we propose a meta-path and meta-structure integrated heterogeneous graph neural network through attention mechanisms (PSHGAN). PSHGAN first maps features of heterogeneous nodes into the same space. Then, PSHGAN learns the weights of two nodes at each end of the meta-path or meta-structure by a local attention mechanism. Finally, PSHGAN learns weights of meta-paths and meta-structures by a global attention mechanism and aggregates the nodes representations. Extensive experiments are conducted on real-world benchmark datasets and show that our proposed model outperforms the state-of-the-art models in the node classification and link prediction tasks. Moreover, we make comprehensive analysis on the impacts of meta-structures on the performance of classification training with meta-paths.

An Experimental Evaluation of Similarity-Based and Embedding-Based Link Prediction Methods on Graphs

The task of inferring missing links or predicting future ones in a graph based on its current structure is referred to as link prediction. Link prediction methods that are based on pairwise node similarity are well-established approaches in the literature and show good prediction performance in many realworld graphs though they are heuristic. On the other hand, graph embedding approaches learn lowdimensional representation of nodes in graph and are capable of capturing inherent graph features, and thus support the subsequent link prediction task in graph. This paper studies a selection of methods from both categories on several benchmark (homogeneous) graphs with different properties from various domains. Beyond the intra and inter category comparison of the performances of the methods, our aim is also to uncover interesting connections between Graph Neural Network(GNN)- based methods and heuristic ones as a means to alleviate the black-box well-known limitation.

An Experimental Evaluation of Similarity-Based and Embedding-Based Link Prediction Methods on Graphs

The task of inferring missing links or predicting future ones in a graph based on its current structure is referred to as link prediction. Link prediction methods that are based on pairwise node similarity are well-established approaches in the literature and show good prediction performance in many realworld graphs though they are heuristic. On the other hand, graph embedding approaches learn lowdimensional representation of nodes in graph and are capable of capturing inherent graph features, and thus support the subsequent link prediction task in graph. This paper studies a selection of methods from both categories on several benchmark (homogeneous) graphs with different properties from various domains. Beyond the intra and inter category comparison of the performances of the methods, our aim is also to uncover interesting connections between Graph Neural Network(GNN)- based methods and heuristic ones as a means to alleviate the black-box well-known limitation.

NSL2CD: identifying potential circRNA-disease associations based on network embedding and subspace learning.

Many studies have evidenced that circular RNAs (circRNAs) are important regulators in various pathological processes and play vital roles in many human diseases, which could serve as promising biomarkers for disease diagnosis, treatment and prognosis. However, the functions of most of circRNAs remain to be unraveled, and it is time-consuming and costly to uncover those relationships between circRNAs and diseases by conventional experimental methods. Thus, identifying candidate circRNAs for human diseases offers new opportunities to understand the functional properties of circRNAs and the pathogenesis of diseases. In this study, we propose a novel network embedding-based adaptive subspace learning method (NSL2CD) for predicting potential circRNA-disease associations and discovering those disease-related circRNA candidates. The proposed method first calculates disease similarities and circRNA similarities by fully utilizing different data sources and learns low-dimensional node representations with network embedding methods. Then, we adopt an adaptive subspace learning model to discover potential associations between circRNAs and diseases. Meanwhile, an integrated weighted graph regularization term is imposed to preserve local geometric structures of data spaces, and L1,2-norm constraint is also incorporated into the model to realize the smoothness and sparsity of projection matrices. The experiment results show that NSL2CD achieves comparable performance under different evaluation metrics, and case studies further confirm its ability to discover potential candidate circRNAs for human diseases.

Attributed Network Embedding with Micro-Meso Structure

Recently, network embedding has received a large amount of attention in network analysis. Although some network embedding methods have been developed from different perspectives, on one hand, most of the existing methods only focus on leveraging the plain network structure, ignoring the abundant attribute information of nodes. On the other hand, for some methods integrating the attribute information, only the lower-order proximities (e.g., microscopic proximity structure) are taken into account, which may suffer if there exists the sparsity issue and the attribute information is noisy. To overcome this problem, the attribute information and mesoscopic community structure are utilized. In this article, we propose a novel network embedding method termed Attributed Network Embedding with Micro-Meso structure, which is capable of preserving both the attribute information and the structural information including the microscopic proximity structure and mesoscopic community structure. In particular, both the microscopic proximity structure and node attributes are factorized by Nonnegative Matrix Factorization (NMF), from which the low-dimensional node representations can be obtained. For the mesoscopic community structure, a community membership strength matrix is inferred by a generative model (i.e., BigCLAM) or modularity from the linkage structure, which is then factorized by NMF to obtain the low-dimensional node representations. The three components are jointly correlated by the low-dimensional node representations, from which two objective functions (i.e., ANEM_B and ANEM_M) can be defined. Two efficient alternating optimization schemes are proposed to solve the optimization problems. Extensive experiments have been conducted to confirm the superior performance of the proposed models over the state-of-the-art network embedding methods.

Role-based network embedding via structural features reconstruction with degree-regularized constraint

Role-based network embedding aims to map network into low-dimensional node representations while preserving structural similarities. Adjacency matrix contain both the local and global information of a network, but it cannot directly represent the role of nodes. So it is essential to extract higher-order structural features from adjacency matrix for role-based network embedding (structural equivalence). While being sensitive to noise in real networks in general, the features extracted by some identical methods cannot truly represent the role of nodes. Therefore, we propose a deep learning framework RESD. In detail, we first propose extracting higher-order structural features for each node in the network. Then, we utilize the Variational Auto-Encoder (VAE) to model the nonlinear relationship of the features, reduce noise and improve the robustness of embedding. Furthermore, in the embedding space, we apply a degree-regularized constraint to guide the representation learning for preserving key structural information of nodes (e.g., degrees), which may be lost due to the principle of VAE framework. Finally, we construct a unified objective function to learn the node embedding for role discovery by preserving the structural features and node degree. We compare our model with several state-of-the-art methods on real-world networks. The results of extensive experiments demonstrate the effectiveness of our model and prove that our model scales well with dimension and network size.

Self-supervised graph convolutional clustering by preserving latent distribution

Node clustering has attracted widespread concern due to the fact that graph structure helps to seek the geometric structure of data. Existing graph convolutional networks (GCNs) based node clustering methods usually contain two steps. First, learning latent nodes representations. Second, inferring the unknown labels. Despite the promising preliminary results, it cannot take full advantage of the information embedded in current pseudo clustering labels, resulting in suboptimal performance. In fact, it is the truth we do not know the true label of the node in clustering, but the pseudo clustering label of the node generated by each iteration is known. Under this condition, although the pseudo labels of some nodes are incorrect, the partial correct label information is useful and significant. Moreover, GCNs based node clustering aims to learn satisfactory low dimensional node representations which are usually used for clustering. However, existing GCNs based clustering methods fail to take into account distribution consistency between the raw data space and the latent space of nodes representations, resulting in insufficient representations. To address the above problems, we propose a novel GCN-based node clustering method. By introducing a self-supervision module, it can employ the good property of pseudo clustering labels to self-supervise the learning of node representations. Moreover, a latent distribution preserving term, which is measured by KL divergence, is employed to help the latent representations of the same sample to further have a consistent distribution in dimension space as well as in the original dimension space. We formulate the above two concerns into a unified optimization framework. Experimental results on several public datasets indicate our method is effective compared with the state-of-the-art node clustering methods.

Adversarial Representation Mechanism Learning for Network Embedding

Network embedding which is to learn a low dimensional representation of nodes in a network has been used in many network analysis tasks. Recently some Generative Adversarial Networks (GAN) based network embedding methods have been proposed. These methods typically use GAN to force the network embedding results to follow a priori distribution (e.g. Gaussian distribution in most cases), which do not make full use of the essential advantage of adversarial learning. To address this problem, we propose a novel adversarial learning framework with three players, which applies the adversarial learning strategy on the representation mechanism. In the new framework, besides the discriminator, there are two other players, named positive sample generator and competitor, which aim to learn two different representation mechanisms. These two players compete with each other to improve their representation mechanisms. Furthermore, to design a competitive competitor, we use the same framework as the positive sample generator but take as input a fake network generated by a designed perturbation strategy. The experimental results show the superior performance of the proposed approach over the existing methods (including the state-of-the-art of embedding method DGI) on a variety of network analysis tasks including node clustering, node classification, link prediction and visualization.

Semantic-Aware Graph Convolutional Networks for Clinical Auxiliary Diagnosis and Treatment of Traditional Chinese Medicine

Traditional Chinese Medicine (TCM) clinical informatization focuses on serving user-oriented health knowledge and facilitating online diagnosis. Regularities are hidden in clinical knowledge play a significant role in the improvement of the TCM informatization service. However, many regularities can hardly be discovered because of specific data-challenges in TCM prescriptions at present. Therefore, in this article, we propose an end-to-end model, called Semantic-aware Graph Convolutional Networks (SaGCN) model, to learn the latent regularities in three steps: (1) We first construct a heterogeneous graph based on prescriptions; (2) We stack Semantic-aware graph convolution to learn effective low-dimensional representations of nodes by meta-graphs and self-attention; (3) With the learned representations, we can detect regularities accurately by clustering and linked prediction. To the best of our knowledge, this is the first study to use metagraph and graph convolutional networks for modeling TCM clinical data and diagnosis prediction. Experimental results on three real datasets demonstrate SaGCN outperforms the state-of-the-art models for clinical auxiliary diagnosis and treatment.

Heterogeneous Information Network Embedding with Adversarial Disentangler

Heterogeneous information network (HIN) embedding has gained considerable attention in recent years, which learns low-dimensional representation of nodes while preserving the semantic and structural correlations in HINs. Many of existing methods which exploit meta-path guided strategy have shown promising results. However, the learned node representations could be highly entangled for downstream tasks; for example, an author's publications in multidisciplinary venues may make the prediction of his/her research interests difficult. To address this issue, we develop a novel framework named HEAD (i.e., HIN Embedding with Adversarial Disentangler) to separate the distinct, informative factors of variations in node semantics formulated by meta-paths. More specifically, in HEAD, we first propose the meta-path disentangler to separate node embeddings from various meta-paths into intrinsic and specific spaces; then with meta-path schemes as self-supervised information, we design two adversarial learners (i.e., meta-path and semantic discriminators) to make the intrinsic embedding more independent from the designed meta-paths while the specific embedding more meta-path dependent. To comprehensively evaluate the performance of HEAD, we perform a set of experiments on four real-world datasets. Compared to the state-of-the-art baselines, the maximum 15% improvement of performance demonstrates the effectiveness of HEAD and the benefits of the learned disentangled representations.

NECo: A node embedding algorithm for multiplex heterogeneous networks.

Complex diseases such as hypertension, cancer, and diabetes cause nearly 70% of the deaths in the U.S. and involve multiple genes and their interactions with environmental factors. Therefore, identification of genetic factors to understand and decrease the morbidity and mortality from complex diseases is an important and challenging task. With the generation of an unprecedented amount of multi-omics datasets, network-based methods have become popular to represent the multilayered complex molecular interactions. Particularly node embeddings, the low-dimensional representations of nodes in a network are utilized for gene function prediction. Integrated network analysis of multi-omics data alleviates the issues related to missing data and lack of context-specific datasets. Most of the node embedding methods, however, are unable to integrate multiple types of datasets from genes and phenotypes. To address this limitation, we developed a node embedding algorithm called Node Embeddings of Complex networks (NECo) that can utilize multilayered heterogeneous networks of genes and phenotypes. We evaluated the performance of NECo using genotypic and phenotypic datasets from rat (Rattus norvegicus) disease models to classify hypertension disease-related genes. Our method significantly outperformed the state-of-the-art node embedding methods, with AUC of 94.97% compared 85.98% in the second-best performer, and predicted genes not previously implicated in hypertension.

SNEGAN: Signed Network Embedding by Using Generative Adversarial Nets

Network embedding (NE) aims to learn low-dimensional node representations of networks while preserving essential node structures and properties. Existing NE methods mainly preserve simple link structures in unsigned networks, neglecting conflicting relationships that widely exist in social media and Internet of things. In this paper, we propose a novel generative adversarial nets learning framework (called SNEGAN) for signed network embedding, which tries to preserve link structures signed by positive or negative labels. SNEGAN combines a generator with a signed random walker technique and a graph softmax function to generate fake links that are used to deceive discriminator. Moreover, it combines a discriminator with a tanh function to discriminate truths and signs of links sampled from the generator and real network structures. The generator and discriminator play a two-player minimax game, and they will eventually generate low-dimensional node representations of signed networks. Extensive experiments on both LFR benchmark and real-world signed networks show the superiority of SNEGAN over the state-of-the-art NE methods in tackling both link (sign) prediction and reconstruction tasks.

Feature Extraction and Representation of Urban Road Networks Based on Travel Routes

Extraction of traffic features constitutes a key research direction in traffic safety planning. In previous traffic tasks, road network features are extracted manually. In contrast, Network Representation Learning aims to automatically learn low-dimensional node representations. Enlightened by feature learning in Natural Language Processing, representation learning of urban nodes is studied as a supervised task in this paper. Following this line of thinking, a deep learning framework, called StreetNode2VEC, is proposed for learning feature representations for nodes in the road network based on travel routes, and then model parameter calibration is performed. We explain the effectiveness of features from visualization, similarity analysis, and link prediction. In visualization, the features of nodes naturally present a clustered pattern, and different clusters correspond to different regions in the road network. Meanwhile, the features of nodes still retain their spatial information in similarity analysis. The proposed method StreetNode2VEC obtains a AUC score of 0.813 in link prediction, which is greater than that obtained from Graph Convolutional Network (GCN) and Node2vec. This suggests that the features of nodes can be used to effectively and credibly predict whether a link should be established between two nodes. Overall, our work provides a new way of representing road nodes in the road network, which have potential in the traffic safety planning field.

Semisupervised Community Preserving Network Embedding with Pairwise Constraints

Network embedding aims to learn the low-dimensional representations of nodes in networks. It preserves the structure and internal attributes of the networks while representing nodes as low-dimensional dense real-valued vectors. These vectors are used as inputs of machine learning algorithms for network analysis tasks such as node clustering, classification, link prediction, and network visualization. The network embedding algorithms, which considered the community structure, impose a higher level of constraint on the similarity of nodes, and they make the learned node embedding results more discriminative. However, the existing network representation learning algorithms are mostly unsupervised models; the pairwise constraint information, which represents community membership, is not effectively utilized to obtain node embedding results that are more consistent with prior knowledge. This paper proposes a semisupervised modularized nonnegative matrix factorization model, SMNMF, while preserving the community structure for network embedding; the pairwise constraints (must-link and cannot-link) information are effectively fused with the adjacency matrix and node similarity matrix of the network so that the node representations learned by the model are more interpretable. Experimental results on eight real network datasets show that, comparing with the representative network embedding methods, the node representations learned after incorporating the pairwise constraints can obtain higher accuracy in node clustering task and the results of link prediction, and network visualization tasks indicate that the semisupervised model SMNMF is more discriminative than unsupervised ones.

Structural Deep Nonnegative Matrix Factorization for community detection

Due to the important role in analyzing the topological structure of complex networks, community detection has attracted increasing attention recently. The network embedding methods have shown promising performances in community detection, aiming to learn low-dimensional representations of nodes in networks. Among them, Nonnegative Matrix Factorization (NMF) is proved to be an efficient approach. However, considering that the mapping between the original network and the community membership space contains rather complex hierarchical information, classic shallow-NMF approaches often fail to capture the complex underlying network structure, resulting in sub-optimal network representations. Therefore, how to find a method that is able to effectively capture the complex underlying network structure and preserve the global and local structure is an open and important topic. Inspired by the excellent ability of representation learning in deep autoencoder, we propose a Structural Deep Nonnegative Matrix Factorization model, named SDNMF, for community detection. Similar to deep autoencoder, the proposed multi-layer NMF-based model consists of a decoder module and an encoder module. Further, we propose to exploit the first-order similarity and second-order similarity jointly to preserve the structural information. The first-order similarity characterizes the local network information. Meanwhile, the global network information can be captured and the sparsity problem is alleviated by the second-order similarity. An efficient learning algorithm is developed to optimize the proposed DANMF model, which simultaneously optimizes the first-order and second-order similarity. The effectiveness of the proposed SDNMF is verified by comparing it with several state-of-the-art approaches for community detection on six real-word networks. The comparison is based on three performance metrics. Experimental results demonstrate that the proposed SDNMF can obtain a more accurate community membership matrix compared with the baselines. The convergence analysis is also performed to verify the efficiency of our approach.

MGAT: Multi-view Graph Attention Networks

Multi-view graph embedding is aimed at learning low-dimensional representations of nodes that capture various relationships in a multi-view network, where each view represents a type of relationship among nodes. Multitudes of existing graph embedding approaches concentrate on single-view networks, that can only characterize one simple type of proximity relationships among objects. However, most of the real-world complex systems possess multiple types of relationships among entities. In this paper, a novel approach of graph embedding for multi-view networks is proposed, named Multi-view Graph Attention Networks (MGAT). We explore an attention-based architecture for learning node representations from each single view, the network parameters of which are constrained by a novel regularization term. In order to collaboratively integrate multiple types of relationships in different views, a view-focused attention method is explored to aggregate the view-wise node representations. We evaluate the proposed algorithm on several real-world datasets, and it demonstrates that the proposed approach outperforms existing state-of-the-art baselines.

Enhanced Unsupervised Graph Embedding via Hierarchical Graph Convolution Network

Graph embedding aims to learn the low-dimensional representation of nodes in the network, which has been paid more and more attention in many graph-based tasks recently. Graph Convolution Network (GCN) is a typical deep semisupervised graph embedding model, which can acquire node representation from the complex network. However, GCN usually needs to use a lot of labeled data and additional expressive features in the graph embedding learning process, so the model cannot be effectively applied to undirected graphs with only network structure information. In this paper, we propose a novel unsupervised graph embedding method via hierarchical graph convolution network (HGCN). Firstly, HGCN builds the initial node embedding and pseudo-labels for the undirected graphs, and then further uses GCNs to learn the node embedding and update labels, finally combines HGCN output representation with the initial embedding to get the graph embedding. Furthermore, we improve the model to match the different undirected networks according to the number of network node label types. Comprehensive experiments demonstrate that our proposed HGCN and HGCN∗ can significantly enhance the performance of the node classification task.