Network Analysis Tasks Research Articles

Dynamic link prediction by learning the representation of node-pair via graph neural networks

Many real-world networks are dynamic, whose structure keeps changing over time. Link prediction, which can foretell the emergence of future links, is one crucial task in dynamic network analysis. Compared to link prediction in static networks, it is more challenging and complicated in dynamic ones due to the dynamic nature. On the other hand, effective use of the information carried out by dynamic networks can enhance prediction accuracy. In this study, we presents a new end-to-end solution for dynamic link prediction, in which the representations of node-pairs can be effectively learned via an improved graph neural network and a nonlinear function by leveraging the structural information of individual snapshots, historical features from network evolution, and global knowledge of the collapsed network. The proposed method can effectively cope with the challenge of dynamic link prediction. Extensive tests are implemented on several dynamic networks to assess the prediction performance of our proposed method. The results on these networks demonstrate that our proposed method achieves superior effectiveness compared to the baselines in most cases.

Deep autoencoder architecture with outliers for temporal attributed network embedding

Temporal attributed network embedding aspires to learn a low-dimensional vector representation for each node in each snapshot of a temporal network, which can be capable of various network analysis tasks such as link prediction and node classification. In temporal attributed networks, attribute similarities or link structures of certain nodes may deviate from the regular nodes of the community they belong to, which are called community outlier nodes. However, many existing embedding methods consider only the link structures and their attributes of the nodes adhere to the community structure of the network while ignoring outlier nodes, this can affect the embedding performance of the regular nodes. In this paper, we propose a temporal attributed network embedding framework with outliers, based on autoencoders, to solve the problem. In particular, we propose an outlier-aware autoencoder to model the node information, which combines the current snapshot and previous snapshots to jointly learn embedded vectors of nodes in the current snapshot of a temporal network. In feature preprocessing, we propose a simplified higher graph convolutional mechanism to incorporate attribute information into link structure information, which can leverage attribute features into link structure. Experimental results on node classification and link prediction reveal that our model is competitive against various baseline models.

Role-aware random walk for network embedding

Network embedding is a fundamental part of many network analysis tasks, including node classification and link prediction. The existing random walk-based embedding methods aim to learn node embedding that preserves information on either node proximity or structural similarity. However, the information on both role and community is important to network nodes. To address the shortcomings of the existing methods, this paper proposes a novel method for network embedding called the RARE, which can be used for the analysis of different types of networks and even disconnected networks. The proposed method uses the role and community information of nodes to preserve both node proximity and structural similarity in the learned node embeddings. The walks generated through the role-aware random walk can capture the role and community information of nodes. The obtained walks are input to the Skip-gram model to learn the final embedding of nodes. In addition, the RARE is extended to the CRARE that adds the sampling of high-order community members to the customized random walk so that the node’s representation can preserve more structural information of the network. The performances of the proposed methods are evaluated on multi-class node classification, link prediction, and network visualization tasks. Experimental results on different domain datasets indicate that the proposed methods outperform the baseline methods. The proposed methods can be further accelerated using parallelization in the random walk generation process. The source code: https://github.com/HeguiZhang/RARE.

Bayesian inference of transition matrices from incomplete graph data with a topological prior

Many network analysis and graph learning techniques are based on discrete- or continuous-time models of random walks. To apply these methods, it is necessary to infer transition matrices that formalize the underlying stochastic process in an observed graph. For weighted graphs, where weighted edges capture observations of repeated interactions between nodes, it is common to estimate the entries of such transition matrices based on the (relative) weights of edges. However in real-world settings we are often confronted with incomplete data, which turns the construction of the transition matrix based on a weighted graph into an inference problem. Moreover, we often have access to additional information, which capture topological constraints of the system, i.e. which edges in a weighted graph are (theoretically) possible and which are not. Examples include transportation networks, where we may have access to a small sample of passenger trajectories as well as the physical topology of connections, or a limited set of observed social interactions with additional information on the underlying social structure. Combining these two different sources of information to reliably infer transition matrices from incomplete data on repeated interactions is an important open challenge, with severe implications for the reliability of downstream network analysis tasks.Addressing this issue, we show that including knowledge on such topological constraints can considerably improve the inference of transition matrices, especially in situations where we only have a small number of observed interactions. To this end, we derive an analytically tractable Bayesian method that uses repeated interactions and a topological prior to perform data-efficient inference of transition matrices. We compare our approach against commonly used frequentist and Bayesian approaches both in synthetic data and in five real-world datasets, and we find that our method recovers the transition probabilities with higher accuracy. Furthermore, we demonstrate that the method is robust even in cases when the knowledge of the topological constraint is partial. Lastly, we show that this higher accuracy improves the results for downstream network analysis tasks like cluster detection and node ranking, which highlights the practical relevance of our method for interdisciplinary data-driven analyses of networked systems.

Reconstruction of random geometric graphs: Breaking the [formula omitted] distortion barrier

Embedding graphs in a geographical or latent space, i.e. inferring locations for vertices in Euclidean space or on a smooth manifold or submanifold, is a common task in network analysis, statistical inference, and graph visualization. We consider the classic model of random geometric graphs where n points are scattered uniformly in a square of area n, and two points have an edge between them if and only if their Euclidean distance is less than r. The reconstruction problem then consists of inferring the vertex positions, up to the symmetries of the square, given only the adjacency matrix of the resulting graph. We give an algorithm that, if r=nα for any 0<α<1/2, with high probability reconstructs the vertex positions with a maximum error of O(nβ) where β=1/2−(4/3)α, until α≥3/8 where β=0 and the error becomes O(logn). This improves over earlier results, which were unable to reconstruct with error less than r. Our method estimates Euclidean distances using a hybrid of graph distances and short-range estimates based on the number of common neighbors. We extend our results to the surface of the sphere in R3 and to hypercubes in any constant fixed dimension.

Billion-Scale Bipartite Graph Embedding: A Global-Local Induced Approach

Bipartite graph embedding (BGE), as the fundamental task in bipartite network analysis, is to map each node to compact low-dimensional vectors that preserve intrinsic properties. The existing solutions towards BGE fall into two groups: metric-based methods and graph neural network-based (GNN-based) methods. The latter typically generates higher-quality embeddings than the former due to the strong representation ability of deep learning. Nevertheless, none of the existing GNN-based methods can handle billion-scale bipartite graphs due to the expensive message passing or complex modelling choices. Hence, existing solutions face a challenge in achieving both embedding quality and model scalability. Motivated by this, we propose a novel graph neural network named AnchorGNN based on global-local learning framework, which can generate high-quality BGE and scale to billion-scale bipartite graphs. Concretely, AnchorGNN leverages a novel anchor-based message passing schema for global learning, which enables global knowledge to be incorporated to generate node embeddings. Meanwhile, AnchorGNN offers an efficient one-hop local structure modelling using maximum likelihood estimation for bipartite graphs with rational analysis, avoiding large adjacency matrix construction. Both global information and local structure are integrated to generate distinguishable node embeddings. Extensive experiments demonstrate that AnchorGNN outperforms the best competitor by up to 36% in accuracy and achieves up to 28 times speed-up against the only metric-based baseline on billion-scale bipartite graphs.

Fuzzy Representation Learning on Graph

Recent years have witnessed a drastic surge in graph representation learning, which usually produces low-dimensional and crisp representations from graph topology and high-dimensional node attributes. Nevertheless, a crisp representation of a node or graph actually conceals the uncertainty and interpretability of features. In citation networks, for example, the reference between two papers is always involved with fuzziness denoting the correlation degrees, that is, one connection may simultaneously belong to strong and weak references in different beliefs. The uncertainty in node connections and attributes inspires us to delve into fuzzy representations. This paper, for the first time, proposes an unsupervised fuzzy representation learning model for graphs and networks to improve their expressiveness by making crisp representations fuzzy. Specifically, we develop a fuzzy graph convolution neural network (FGCNN), which could aggregate high-level fuzzy features, leveraging fuzzy logic to fully excavate feature-level uncertainties, and finally generate fuzzy representations. The corresponding hierarchical model composed of multiple FGCNNs, called deep fuzzy graph convolution neural network (DFGCNN), is able to generate fuzzy node representations which are more expressive than crisp ones. Experimental results of multiple network analysis tasks validate that the proposed fuzzy representation models have strong competitiveness against the state-of-the-art baselines over several real-world datasets.

Multiplex heterogeneous network representation learning with unipath based global awareness neural network

Network embedding has gained great popularity in tackling various network analytical tasks, such as link prediction and node classification. However, most existing works from heterogeneous networks ignore the relation heterogeneity with multiplex networks between multi-typed nodes. To tackle this challenge, this work proposes a Unipath based global Awareness neural Network (UAN) for attributed multiplex heterogeneous network embedding. Our UAN can automatically learn useful interactions of unipath networks and the base network. Additionally, we effectively integrate both multi-relation structural signals and attribute semantics into the learned node embeddings with both unsupervised and semi-supervised learning paradigms. Extensive experiments were conducted on real-world datasets in three different domains and various network analysis tasks were performed. Experimental results demonstrate the significant superiority of UAN against state-of-the-art embedding baselines in terms of all evaluation metrics.

Benchmarking graph representation learning algorithms for detecting modules in molecular networks

Background: A common task in molecular network analysis is the detection of community structures or modules. Such modules are frequently associated with shared biological functions and are often disrupted in disease. Detection of community structure entails clustering nodes in the graph, and many algorithms apply a clustering algorithm on an input node embedding. Graph representation learning offers a powerful framework to learn node embeddings to perform various downstream tasks such as clustering. Deep embedding methods based on graph neural networks can have substantially better performance on machine learning tasks on graphs, including module detection; however, existing studies have focused on social and citation networks. It is currently unclear if deep embedding methods offer any advantage over shallow embedding methods for detecting modules in molecular networks. Methods: Here, we investigated deep and shallow graph representation learning algorithms on synthetic and real cell-type specific gene interaction networks to detect gene modules and identify pathways affected by sequence nucleotide polymorphisms. We used multiple criteria to assess the quality of the clusters based on connectivity as well as overrepresentation of biological processes. Results: On synthetic networks, deep embedding based on a variational graph autoencoder had superior performance as measured by modularity metrics, followed closely by shallow methods, node2vec and Graph Laplacian embedding. However, the performance of the deep methods worsens when the overall connectivity between clusters increases. On real molecular networks, deep embedding methods did not have a clear advantage and the performance depended upon the properties of the graph and the metrics. Conclusions: Deep graph representation learning algorithms for module detection-based tasks can be beneficial for some biological networks, but the performance depends upon the metrics and graph properties. Across different network types, Graph Laplacian embedding followed by node2vec are the best performing algorithms.

Integrating Node Importance and Network Topological Properties for Link Prediction in Complex Network

Link prediction is one of the most important and challenging tasks in complex network analysis, which aims to predict the existence of unknown links based on the known information in the network. As critical topological properties in the network, node’s degree and clustering coefficient are well-suited for describing the tightness of connection between nodes. The importance of node can affect the possibility of link existence to a certain extent. By analyzing the impact of different centrality on links, which concluded that the degree centrality and proximity centrality have the greatest influence on network link prediction. A link prediction algorithm combines importance of node and network topological properties, called DCCLP, is proposed in this paper, the symmetry of the adjacency matrix is considered in the DCCLP link prediction algorithm to further describe the structural similarity of network nodes. In the training phase of the DCCLP algorithm, the maximized AUC indicator in the training set as the objective, and the optimal parameters are estimated by utilizing the White Shark Optimization algorithm. Then the prediction accuracy of the DCCLP algorithm is evaluated in the test set. By experimenting on twenty-one networks with different scales, and comparing with existing algorithms, the experimental results show that the effectiveness and feasibility of DCCLP algorithm, and further illustrate the importance of the degree centrality of node pairs and proximity centrality of nodes to improve the prediction accuracy of link prediction.

Multi-View Learning-Based Fast Edge Embedding for Heterogeneous Graphs

Edge embedding is a technique for constructing low-dimensional feature vectors of edges in heterogeneous graphs, which are also called heterogeneous information networks (HINs). However, edge embedding research is still in its early stages, and few well-developed models exist. Moreover, existing models often learn features on the edge graph, which is much larger than the original network, resulting in slower speed and inaccurate performance. To address these issues, a multi-view learning-based fast edge embedding model is developed for HINs in this paper, called MVFEE. Based on the “divide and conquer” strategy, our model divides the global feature learning into multiple separate local intra-view features learning and inter-view features learning processes. More specifically, each vertex type in the edge graph (each edge type in HIN) is first treated as a view, and a private skip-gram model is used to rapidly learn the intra-view features. Then, a cross-view learning strategy is designed to further learn the inter-view features between two views. Finally, a multi-head attention mechanism is used to aggregate these local features to generate accurate global features of each edge. Extensive experiments on four datasets and three network analysis tasks show the advantages of our model.

Amer: A New Attribute-Missing Network Embedding Approach.

Network embedding which aims to learn a low dimensional representation of nodes is a powerful technique for network analysis. While network embedding for networks with complete attributes has been widely investigated, in many real-world applications the attributes of partial nodes are unobserved (i.e., missing) due to privacy concern or resource limit. Very recently, several network embedding methods have been proposed for attribute-missing networks. They first complete the missing attributes and then use the complemented network to learn network embedding. The parameters of these two processes cannot be adjusted by each other, resulting in compromised results. To address this problem, we propose a unified model in which the process of completing missing attributes and the process of learning embedding are not separated but closely intertwined. Being specific, completing missing attributes is under the guidance of learning network representation via mutual information maximization, and the complemented attributes directly enter network representation module which will generate further feedback for completing missing attributes. We further impose attribute-structure relationship constraint for completing missing attributes by designing a new generative adversarial networks (GANs) model. To the best of our knowledge, this is the first unified model for attribute-missing network embedding. Empirical results on real-world datasets show the superiority of our new method over other state-of-the-art methods on four network analysis tasks, including node classification, node clustering, link prediction, and network visualization.

Attribute Augmented Network Embedding Based on Generative Adversarial Nets.

Network embedding is to learn low-dimensional representations of nodes while preserving necessary information for network analysis tasks. Though representations preserving both structure and attribute features have achieved in many real-world applications, learning these representations for networks with attribute information is difficult due to the heterogeneity between structure and attribute information. Many existing methods have been proposed to preserve explicit proximities between nodes, with optimization limited to node pairs with large structure and attribute proximities, which may lead to overfitting. To address the above problems, we adopt an attribute augmented network to represent attribute and structure information in a unified framework. Specifically, we study the problem of attribute augmented network embedding that exploits the strength of generative adversarial nets (ANGANs) in capturing the latent distribution of data to learn robust and informative representations of nodes. The ANGAN method obtains the low-dimensional representations of nodes through adversarial learning between the generative and discriminative models. The generative model approximates the underlying connectivity and attributes distributions of nodes by using the distributions generated from the learned representations. It is implemented by utilizing the properties of the attribute augmented network to improve the traditional Skip-gram model. The discriminative model is designed as a binary classifier to distinguish the truly connected node pairs from the generated ones. The pre-training algorithm and the teacher forcing approach are adopted to improve training efficiency and stability. Empirical results show that ANGAN generally outperforms state-of-the-art methods in various real-world applications, which demonstrates the effectiveness and generality of our method.

The State of the Art in Visualizing Dynamic Multivariate Networks

AbstractMost real‐world networks are both dynamic and multivariate in nature, meaning that the network is associated with various attributes and both the network structure and attributes evolve over time. Visualizing dynamic multivariate networks is of great significance to the visualization community because of their wide applications across multiple domains. However, it remains challenging because the techniques should focus on representing the network structure, attributes and their evolution concurrently. Many real‐world network analysis tasks require the concurrent usage of the three aspects of the dynamic multivariate networks. In this paper, we analyze current techniques and present a taxonomy to classify the existing visualization techniques based on three aspects: temporal encoding, topology encoding, and attribute encoding. Finally, we survey application areas and evaluation methods; and discuss challenges for future research.

Robust and fast representation learning for heterogeneous information networks

Network representation learning is an important tool that can be used to optimize the speed and performance of downstream analysis tasks by extracting latent features of heterogeneous networks. However, in the face of new challenges of increasing network size, diverse latent features, and unseen network noise, existing representation models need to be further optimized. In this paper, a robust and fast representation learning model is proposed for heterogeneous networks, called RFRL. First, the global features of a heterogeneous network are divided into multiple intra-type local features and inter-type local features, and a type-aware biased sampling is designed to generate training samples for each local feature. Second, a node-type-aware and a link-type-aware shallow representation strategy are used to learn intra-type features and inter-type features respectively. This enables the model to achieve good performance while having high speed through the divide-and-conquer learning process and shallow learning model, thus coping with increasing network size and latent feature diversity. Finally, adversarial learning is used to integrate the above two representation strategies to address unseen network noise and enhance the robustness of representation learning. Extensive experiments on three network analysis tasks and three public datasets demonstrate the good performance of our RFRL model.

Dual-channel embedding learning model for partially labeled attributed networks

Network embedding is an important fundamental work in many network application tasks, which encodes the input network from the high-dimensional and sparse topological space into a low-dimensional and dense vector space. Recently, there has been a growing interest in embedding learning on Partially Labeled Attributed Networks (PLANs) due to the increasing occurrence of node attributes and partially available category labels in real-world networks. Semi-supervised embedding learning is a standard approach employed in PLANs, utilizing category labels to supervise the learning process. However, the semi-supervised learning procedure can fail when labels are scarce, noisy, or unreliable. Additionally, most existing embedding algorithms have not successfully integrated heterogeneous information, such as labels, attributes, and structure. To address these issues, we develop a new model, the Dual-Channel Network Embedding (DcNE), which integrates different types of network information into embeddings from a mutual information (MI) perspective. Specifically, we construct a dual-channel information propagation framework to encode the input network in semi-supervised and self-supervised learning paradigms in parallel. Furthermore, a redundancy elimination module is implemented to capture and eliminate the redundant information between the two encoders. Finally, we propose a unified optimization model that integrates the two learning paradigms to collaborate effectively. In the experiments, we demonstrate the effectiveness of DcNE in various network analysis tasks using real-world datasets, establishing its superiority over state-of-the-art baselines.

Minimax optimal clustering of bipartite graphs with a generalized power method

Abstract Clustering bipartite graphs is a fundamental task in network analysis. In the high-dimensional regime where the number of rows $n_{1}$ and the number of columns $n_{2}$ of the associated adjacency matrix are of different order, the existing methods derived from the ones used for symmetric graphs can come with sub-optimal guarantees. Due to increasing number of applications for bipartite graphs in the high-dimensional regime, it is of fundamental importance to design optimal algorithms for this setting. The recent work of Ndaoud et al. (2022, IEEE Trans. Inf. Theory, 68, 1960–1975) improves the existing upper-bound for the misclustering rate in the special case where the columns (resp. rows) can be partitioned into $L = 2$ (resp. $K = 2$) communities. Unfortunately, their algorithm cannot be extended to the more general setting where $K \neq L \geq 2$. We overcome this limitation by introducing a new algorithm based on the power method. We derive conditions for exact recovery in the general setting where $K \neq L \geq 2$, and show that it recovers the result in Ndaoud et al. (2022, IEEE Trans. Inf. Theory, 68, 1960–1975). We also derive a minimax lower bound on the misclustering error when $K=L$ under a symmetric version of our model, which matches the corresponding upper bound up to a factor depending on $K$.

A multi-level generative framework for community detection in attributed networks

AbstractCommunity detection in attributed networks is one of the most important tasks in complex network analysis. Many existing methods propose to integrate the network topology and node attribute from a generative aspect, which models an attributed network as a probabilistic generation process with the community distribution described by hidden variables. Though they can provide good interpretability to the community structure, it is difficult to infer community membership quickly due to their high computational complexity when inferring. Motivated by the multi-level strategy, in this study, we propose a multi-level generative framework to reduce the time cost of generative models for community detection in attributed networks. We first coarsen an attributed network into smaller ones by node matching. Then, we employ the existing generative model on the coarsest network without any modification for community detection, thus efficiently obtaining community memberships of nodes in this small coarsest network. Last, we project the assignments back to the original network through a local refinement mechanism to get communities. Extensive experiments on several real-world and artificial attributed networks show that our multi-level-based method is significantly faster than original generative models and is able to achieve better or more competitive results.

Dynamic community detection including node attributes

Community detection is an important task in social network analysis. It is generally based on the links of a static network, where groups of connected nodes can be found. Real-world problems, however, are often characterized by behavior that changes over time. In such cases, we need dynamic community detection algorithms because they better capture the underlying dynamics. Furthermore, the inclusion of node attributes provides a more robust approach since dynamic attribute values could also indicate changes in the communities. We propose an algorithm for COmmunity DEtection in Dynamic Attributed NETworks (CoDeDANet), which allows us to find groups in dynamic attributed networks using both the link and node information. In the first phase, based on spectral clustering, the attributes’ importance is optimized in a setting that joins the nodes’ features with a topological structure. In a second phase, tensors are used to consider not only current but also past information. The algorithm was tested on four synthetic networks and two real-world social networks. The results show that CoDeDANet outperforms the other state-of-the-art community detection algorithms.

Dual adaptive learning multi-task multi-view for graph network representation learning

Graph network analysis, which achieves widely application, is to explore and mine the graph structure data. However, existing graph network analysis methods with graph representation learning technique ignore the correlation between multiple graph network analysis tasks, and they need massive repeated calculation to obtain each graph network analysis results. Or they cannot adaptively balance the relative importance of multiple graph network analysis tasks, that lead to weak model fitting. Besides, most of existing methods ignore multiplex views semantic information and global graph information, which fail to learn robust node embeddings resulting in unsatisfied graph analysis results. To solve these issues, we propose a multi-task multi-view adaptive graph network representation learning model, called M2agl. The highlights of M2agl are as follows: (1) Graph convolutional network with the linear combination of the adjacency matrix and PPMI (positive point-wise mutual information) matrix is utilized as encoder to extract the local and global intra-view graph feature information of the multiplex graph network. Each intra-view graph information of the multiplex graph network can adaptively learn the parameters of graph encoder. (2) We use regularization to capture the interaction information among different graph views, and the importance of different graph views are learned by view attention mechanism for further inter-view graph network fusion. (3) The model is trained oriented by multiple graph network analysis tasks. The relative importance of multiple graph network analysis tasks are adjusted adaptively with the homoscedastic uncertainty. The regularization can be considered as an auxiliary task to further boost the performance. Experiments on real-worlds attributed multiplex graph networks demonstrate the effectiveness of M2agl in comparison with other competing approaches.