Network Representation Learning Research Articles

IMPRL-Net: interpretable multi-view proximity representation learning network

IMPRL-Net: interpretable multi-view proximity representation learning network

A recurrent graph neural network for inductive representation learning on dynamic graphs

Graph representation learning has recently garnered significant attention due to its wide applications in graph analysis tasks. It is well-known that real-world networks are dynamic, with edges and nodes evolving over time. This presents unique challenges that are distinct from those of static networks. However, most graph representation learning methods are either designed for static graphs, or address only partial challenges associated with dynamic graphs. They overlook the intricate interplay between topology and temporality in the evolution of dynamic graphs and the complexity of sequence modeling. Therefore, we propose a new dynamic graph representation learning model, called as R-GraphSAGE, which takes comprehensive considerations for embedding dynamic graphs. By incorporating a recurrent structure into GraphSAGE, the proposed R-GraphSAGE explores structural and temporal patterns integrally to capture more fine-grained evolving patterns of dynamic graphs. Additionally, it offers a lightweight architecture to decrease the computational costs for handling snapshot sequences, achieving a balance between performance and complexity. Moreover, it can inductively process the addition of new nodes and adapt to the situations without labels and node attributes. The performance of the proposed R-GraphSAGE is evaluated across various downstream tasks with both synthetic and real-world networks. The experimental results demonstrate that it outperforms state-of-the-art baselines by a significant margin in most cases.

A graph-based approach for traffic prediction using similarity and causal relations between nodes

Accurate traffic prediction is crucial for the development of intelligent transportation systems (ITS) and various smart applications. To achieve this, effectively capturing the complex interplay between spatial and temporal dependencies across traffic nodes is essential. However, existing research often focuses solely on fixed spatial dependencies between neighboring nodes, neglecting the potential influence of distant nodes. This paper addresses these limitations by exploring two key types of relationships among traffic nodes. First, we examine indirect relationships based on similarity, where nodes exhibit similar traffic patterns irrespective of their physical proximity. Second, we investigate direct causal relationships, where traffic conditions at one node are directly influenced by other nodes. Based on these findings, we propose a novel approach named TPSC (Traffic Prediction using nodes Similarity and Causal relations). TPSC incorporates a classifier that utilizes the K-Prototype algorithm to group nodes based on their traffic similarity and nearby points of interest (POIs). Separate models are then trained for each cluster. Our model leverages a spatial module that employs graph convolutional network representation learning alongside transfer entropy to capture causal relationships and dynamic spatial dependencies. Additionally, a temporal module is introduced to capture periodic temporal dependencies across three components: recent patterns, daily patterns, and weekly patterns. Extensive experiments are conducted to evaluate proposed model on four large-scale traffic datasets for multiple traffic characteristics including speed, flow, and travel time. The experimental results demonstrate that TPSC surpasses the performance of the compared baselines, highlighting its superior predictive capabilities in traffic prediction.

Representation learning of in-degree-based digraph with rich information

AbstractNetwork representation learning aims to map the relationship between network nodes and context nodes to a low-dimensional representation vector space. Directed network representation learning considers mapping directional of node vector. Currently, only sporadic work on direct network representation has been reported. In this work, we propose a novel algorithm that takes into account the direction of the edge with text’s attribute of the node in directed network representation learning. We then define the matrix based on in-degree of Laplacian and signless Laplacian for digraph, and it utilizes web page datasets from universities in the USA to evaluate the performance of vertex classification. We compare our algorithm with other directed representation learning algorithms. The experimental results show that our algorithm outperforms the baseline by over 20% when the training ratio ranges from 10 to 90%. We apply the in-degree-Laplacian and In-degree-signless-Laplacian to directed representation learning, which is one of the main contributions of this algorithm. Additionally, we incorporate text information through matrix completion in directed network representation learning and the experimental results show an increase in performance of up to 20% compared to the baseline, especially when the training ratio is 10%.

Two-stage short-term wind power probabilistic prediction using natural gradient boosting combined with neural network

Accurate wind power probabilistic prediction reflects the uncertainty information of wind power generation, which is the foundation for optimal scheduling of power systems. This study proposes a two-stage probabilistic prediction model combining natural gradient boosting and neural network for accurate uncertainty estimation of short-term output in a wind farm. In the first stage, the selected input features containing historical and future information are fed into a neural network for representation learning. In the second stage, the extracted abstract features are concatenated with the original features, and a natural gradient boosting model is employed to acquire short-term probabilistic forecasts. The experimental results using data from two real wind farms indicate that the proposed hybrid model can generate accurate, sharp, and reliable forecasts. After performing the successive day-ahead prediction task for a month in the first wind farm, the average root mean square error and mean absolute error of the proposed model in the point prediction were 0.1330 and 0.1070, respectively, which were 6.21%–57.29% and 2.96%–62.03% lower than those of comparative models. In addition, the model’s forecasting probability density curves demonstrate high reliability; its coverage probability and the mean width percentage of the interval prediction results under the 90% confidence level were 0.9094 and 0.3696, respectively, which were more suitable than those of five other probabilistic prediction models.

DHONE: Density-based higher-order network embedding

Studies have indicated that focusing solely on pairwise interactions between two nodes disregards the associativity among multi-nodes in the network’s local structure. This associativity can be seen as dependencies among nodes, where certain edges’ presence depends on the path leading to it. Examinations on diverse datasets have approved that the variable order of chained dependencies allows for the preservation of structure information, which enables the reconstruction of the original network into a Higher-Order Network (HON) with improved quality of network representation. This paper proposes a Density-based Higher-Order Network Embedding (DHONE) algorithm, which integrates the concept of higher-order density into the network-embedding process in order to classify the contribution of different orders of dependencies. Through the construction of a novel and effective higher-order adjacency matrix, DHONE steadily improves the accuracy of network representation learning. Experimental results demonstrate DHONEs proficiency in improving embedding accuracy and overall algorithm robustness. Furthermore, grounded in the concept of higher-order density proposed herein, numerous dependencies have been discerned within the network generated from trajectories, potentially indicating the role of multi-node structures in networks.

WalkGAN: Network Representation Learning With Sequence-Based Generative Adversarial Networks.

Network representation learning, also known as network embedding, aims to learn the low-dimensional representations of vertices while capturing and preserving the network structure. For real-world networks, the edges that represent some important relationships between the vertices of a network may be missed and may result in degenerated performance. The existing methods usually treat missing edges as negative samples, thereby ignoring the true connections between two vertices in a network. To capture the true network structure effectively, we propose a novel network representation learning method called WalkGAN, where random walk scheme and generative adversarial networks (GAN) are incorporated into a network embedding framework. Specifically, WalkGAN leverages GAN to generate the synthetic sequences of the vertices that sufficiently simulate random walk on a network and further learn vertex representations from these vertex sequences. Thus, the unobserved links between the vertices are inferred with high probability instead of treating them as nonexistence. Experimental results on the benchmark network datasets demonstrate that WalkGAN achieves significant performance improvements for vertex classification, link prediction, and visualization tasks.

Edge‐guided representation learning for underwater object detection

AbstractUnderwater object detection (UOD) is crucial for marine economic development, environmental protection, and the planet's sustainable development. The main challenges of this task arise from low‐contrast, small objects, and mimicry of aquatic organisms. The key to addressing these challenges is to focus the model on obtaining more discriminative information. The authors observe that the edges of underwater objects are highly unique and can be distinguished from low‐contrast or mimicry environments based on their edges. Motivated by this observation, an Edge‐guided Representation Learning Network, termed ERL‐Net is proposed, that aims to achieve discriminative representation learning and aggregation under the guidance of edge cues. Firstly, an edge‐guided attention module is introduced to model the explicit boundary information, which generates more discriminative features. Secondly, a hierarchical feature aggregation module is proposed to aggregate the multi‐scale discriminative features by regrouping them into three levels, effectively aggregating global and local information for locating and recognising underwater objects. Finally, a wide and asymmetric receptive field block is proposed to enable features to have a wider receptive field, allowing the model to focus on smaller object information. Comprehensive experiments on three challenging underwater datasets show that our method achieves superior performance on the UOD task.

Graph Representation Learning for Large-Scale Neuronal Morphological Analysis.

The analysis of neuronal morphological data is essential to investigate the neuronal properties and brain mechanisms. The complex morphologies, absence of annotations, and sheer volume of these data pose significant challenges in neuronal morphological analysis, such as identifying neuron types and large-scale neuron retrieval, all of which require accurate measuring and efficient matching algorithms. Recently, many studies have been conducted to describe neuronal morphologies quantitatively using predefined measurements. However, hand-crafted features are usually inadequate for distinguishing fine-grained differences among massive neurons. In this article, we propose a novel morphology-aware contrastive graph neural network (MACGNN) for unsupervised neuronal morphological representation learning. To improve the retrieval efficiency in large-scale neuronal morphological datasets, we further propose Hash-MACGNN by introducing an improved deep hash algorithm to train the network end-to-end to learn binary hash representations of neurons. We conduct extensive experiments on the largest dataset, NeuroMorpho, which contains more than 100 000 neurons. The experimental results demonstrate the effectiveness and superiority of our MACGNN and Hash-MACGNN for large-scale neuronal morphological analysis.

CapMax: A Framework for Dynamic Network Representation Learning From the View of Multiuser Communication.

In this article, a modified mutual information maximization (InfoMax) framework, named channel capacity maximization (CapMax), is proposed and applied to learn informative representations for dynamic networks with time-varying topology and/or time-evolving node attributes. The CapMax is based on the network information theory for multiuser communication, where the representation model is treated as a multiaccess communication channel with memory and feedback. Without requirements of the backbone structure, the learning objective of our CapMax is maximizing the channel capacity, which is measured by directed information (DI) rather than mutual information. For efficient implementation, we design an estimator of the channel capacity through the combination of graph neural networks (GNNs) and recurrent neural networks (RNNs). Under some mild conditions, we theoretically prove that DI is a better measure than mutual information in capturing useful information. The experiments are conducted on multiple real-world dynamic network datasets, and the outperformance of our CapMax on different backbone models on link detection and prediction validates the effectiveness of modeling the representation model as a communication channel.

Not All Instances Contribute Equally: Instance-Adaptive Class Representation Learning for Few-Shot Visual Recognition.

Few-shot visual recognition refers to recognize novel visual concepts from a few labeled instances. Many few-shot visual recognition methods adopt the metric-based meta-learning paradigm by comparing the query representation with class representations to predict the category of query instance. However, the current metric-based methods generally treat all instances equally and consequently often obtain biased class representation, considering not all instances are equally significant when summarizing the instance-level representations for the class-level representation. For example, some instances may contain unrepresentative information, such as too much background and information of unrelated concepts, which skew the results. To address the above issues, we propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition. Specifically, we develop an adaptive instance revaluing network (AIRN) with the capability to address the biased representation issue when generating the class representation, by learning and assigning adaptive weights for different instances according to their relative significance in the support set of corresponding class. In addition, we design an improved bilinear instance representation and incorporate two novel structural losses, i.e., intraclass instance clustering loss and interclass representation distinguishing loss, to further regulate the instance revaluation process and refine the class representation. We conduct extensive experiments on four commonly adopted few-shot benchmarks: miniImageNet, tieredImageNet, CIFAR-FS, and FC100 datasets. The experimental results compared with the state-of-the-art approaches demonstrate the superiority of our ICRL-Net.

Anomalous citations detection in academic networks

Citation network analysis attracts increasing attention from disciplines of complex network analysis and science of science. One big challenge in this regard is that there are unreasonable citations in citation networks, i.e., cited papers are not relevant to the citing paper. Existing research on citation analysis has primarily concentrated on the contents and ignored the complex relations between academic entities. In this paper, we propose a novel research topic, that is, how to detect anomalous citations. To be specific, we first define anomalous citations and propose a unified framework, named ACTION, to detect anomalous citations in a heterogeneous academic network. ACTION is established based on non-negative matrix factorization and network representation learning, which considers not only the relevance of citation contents but also the relationships among academic entities including journals, papers, and authors. To evaluate the performance of ACTION, we construct three anomalous citation datasets. Experimental results demonstrate the effectiveness of the proposed method. Detecting anomalous citations carry profound significance for academic fairness.

Quintuple-based Representation Learning for Bipartite Heterogeneous Networks

Recent years have seen rapid progress in network representation learning, which removes the need for burdensome feature engineering and facilitates downstream network-based tasks. In reality, networks often exhibit heterogeneity, which means there may exist multiple types of nodes and interactions. Heterogeneous networks raise new challenges to representation learning, as the awareness of node and edge types is required. In this paper, we study a basic building block of general heterogeneous networks, the heterogeneous networks with two types of nodes. Many problems can be solved by decomposing general heterogeneous networks into multiple bipartite ones. Recently, to overcome the demerits of non-metric measures used in the embedding space, metric learning-based approaches have been leveraged to tackle heterogeneous network representation learning. These approaches first generate triplets of samples, in which an anchor node, a positive counterpart and a negative one co-exist, and then try to pull closer positive samples and push away negative ones. However, when dealing with heterogeneous networks, even the simplest two-typed ones, triplets cannot simultaneously involve both positive and negative samples from different parts of networks. To address this incompatibility of triplet-based metric learning, in this paper, we propose a novel quintuple-based method for learning node representations in bipartite heterogeneous networks. Specifically, we generate quintuples that contain positive and negative samples from two different parts of networks. And we formulate two learning objectives that accommodate quintuple-based learning samples, a proximity-based loss that models the relations in quintuples by sigmoid probabilities, and an angular loss that more robustly maintains similarity structures. In addition, we also parameterize feature learning by using one-dimensional convolution operators around nodes’ neighborhoods. Compared with eight methods, extensive experiments on two downstream tasks manifest the effectiveness of our approach.

Contrastive Balancing Representation Learning for Heterogeneous Dose-Response Curves Estimation

Estimating the individuals' potential response to varying treatment doses is crucial for decision-making in areas such as precision medicine and management science. Most recent studies predict counterfactual outcomes by learning a covariate representation that is independent of the treatment variable. However, such independence constraints neglect much of the covariate information that is useful for counterfactual prediction, especially when the treatment variables are continuous. To tackle the above issue, in this paper, we first theoretically demonstrate the importance of the balancing and prognostic representations for unbiased estimation of the heterogeneous dose-response curves, that is, the learned representations are constrained to satisfy the conditional independence between the covariates and both of the treatment variables and the potential responses. Based on this, we propose a novel Contrastive balancing Representation learning Network using a partial distance measure, called CRNet, for estimating the heterogeneous dose-response curves without losing the continuity of treatments. Extensive experiments are conducted on synthetic and real-world datasets demonstrating that our proposal significantly outperforms previous methods.

Multi-Modal Disordered Representation Learning Network for Description-Based Person Search

Description-based person search aims to retrieve images of the target identity via textual descriptions. One of the challenges for this task is to extract discriminative representation from images and descriptions. Most existing methods apply the part-based split method or external models to explore the fine-grained details of local features, which ignore the global relationship between partial information and cause network instability. To overcome these issues, we propose a Multi-modal Disordered Representation Learning Network (MDRL) for description-based person search to fully extract the visual and textual representations. Specifically, we design a Cross-modality Global Feature Learning Architecture to learn the global features from the two modalities and meet the demand of the task. Based on our global network, we introduce a Disorder Local Learning Module to explore local features by a disordered reorganization strategy from both visual and textual aspects and enhance the robustness of the whole network. Besides, we introduce a Cross-modality Interaction Module to guide the two streams to extract visual or textual representations considering the correlation between modalities. Extensive experiments are conducted on two public datasets, and the results show that our method outperforms the state-of-the-art methods on CUHK-PEDES and ICFG-PEDES datasets and achieves superior performance.

A deep contrastive framework for unsupervised temporal link prediction in dynamic networks

In dynamic networks, temporal link prediction aims to predict the appearance and disappearance of links in future snapshots based on the network structure we have observed. It also plays a crucial role in network analysis and predicting the behavior of the dynamic system. However, most existing studies only focus on supervised temporal link prediction problems, i.e., taking part of the links in future snapshots as supervised information. The ones that can solve the unsupervised temporal link prediction problem are mainly based on matrix decomposition, which lack the capability to automatically extract nonlinear spatial and temporal features from dynamic networks. The most challenging part of this problem is to extract the inherent evolution of the patterns hidden in dynamic networks in unsupervised ways. Inspired by the application and achievement of contrastive learning in network representation learning, we propose a novel deep Contrastive framework for unsupervised Temporal Link Prediction (CTLP). Our framework is based on a deep encoder-decoder architecture, which can capture the nonlinear structure and temporal features automatically and can predict future links of subsequent snapshots of dynamic networks in an unsupervised manner. Besides, CTLP could handle the multi-step temporal link prediction problem of dynamic networks through attenuation modeling across the snapshots. Extensive experiments on temporal link prediction show that our CTLP framework significantly outperforms state-of-the-art unsupervised methods, and even outperforms the supervised methods in some cases.

Contrastive representation learning on dynamic networks

Representation learning for dynamic networks is designed to learn the low-dimensional embeddings of nodes that can well preserve the snapshot structure, properties and temporal evolution of dynamic networks. However, current dynamic network representation learning methods tend to focus on estimating or generating observed snapshot structures, paying excessive attention to network details, and disregarding distinctions between snapshots with larger time intervals, resulting in less robustness for sparse or noisy networks. To alleviate these challenges, this paper proposes a contrastive mechanism for temporal representation learning on dynamic networks, inspired by the success of contrastive learning in visual and static network representation learning. This paper proposes a novel Dynamic Network Contrastive representation Learning (DNCL) model. Specifically, contrast objective functions are constructed using intra-snapshot and inter-snapshot contrasts to capture the network topology, node feature information, and network evolution information, respectively. Rather than estimating or generating ground-truth network features, the proposed approach maximizes mutual information between nodes from different time steps and views generated. The experimental results of link prediction, node classification, and clustering on several real-world and synthetic networks demonstrate the superiority of DNCL over state-of-the-art methods, indicating the effectiveness of the proposed approach for dynamic network representation learning.

A multi-source molecular network representation model for protein–protein interactions prediction

The prediction of potential protein–protein interactions (PPIs) is a critical step in decoding diseases and understanding cellular mechanisms. Traditional biological experiments have identified plenty of potential PPIs in recent years, but this problem is still far from being solved. Hence, there is urgent to develop computational models with good performance and high efficiency to predict potential PPIs. In this study, we propose a multi-source molecular network representation learning model (called MultiPPIs) to predict potential protein–protein interactions. Specifically, we first extract the protein sequence features according to the physicochemical properties of amino acids by utilizing the auto covariance method. Second, a multi-source association network is constructed by integrating the known associations among miRNAs, proteins, lncRNAs, drugs, and diseases. The graph representation learning method, DeepWalk, is adopted to extract the multisource association information of proteins with other biomolecules. In this way, the known protein–protein interaction pairs can be represented as a concatenation of the protein sequence and the multi-source association features of proteins. Finally, the Random Forest classifier and corresponding optimal parameters are used for training and prediction. In the results, MultiPPIs obtains an average 86.03% prediction accuracy with 82.69% sensitivity at the AUC of 93.03% under five-fold cross-validation. The experimental results indicate that MultiPPIs has a good prediction performance and provides valuable insights into the field of potential protein–protein interactions prediction. MultiPPIs is free available at https://github.com/jiboyalab/multiPPIs.

Multi-channel high-order network representation learning research.

The existing network representation learning algorithms mainly model the relationship between network nodes based on the structural features of the network, or use text features, hierarchical features and other external attributes to realize the network joint representation learning. Capturing global features of the network allows the obtained node vectors to retain more comprehensive feature information during training, thereby enhancing the quality of embeddings. In order to preserve the global structural features of the network in the training results, we employed a multi-channel learning approach to perform high-order feature modeling on the network. We proposed a novel algorithm for multi-channel high-order network representation learning, referred to as the Multi-Channel High-Order Network Representation (MHNR) algorithm. This algorithm initially constructs high-order network features from the original network structure, thereby transforming the single-channel network representation learning process into a multi-channel high-order network representation learning process. Then, for each single-channel network representation learning process, the novel graph assimilation mechanism is introduced in the algorithm, so as to realize the high-order network structure modeling mechanism in the single-channel network representation learning. Finally, the algorithm integrates the multi-channel and single-channel mechanism of high-order network structure joint modeling, realizing the efficient use of network structure features and sufficient modeling. Experimental results show that the node classification performance of the proposed MHNR algorithm reaches a good order on Citeseer, Cora, and DBLP data, and its node classification performance is better than that of the comparison algorithm used in this paper. In addition, when the vector length is optimized, the average classification accuracy of nodes of the proposed algorithm is up to 12.24% higher than that of the DeepWalk algorithm. Therefore, the node classification performance of the proposed algorithm can reach the current optimal order only based on the structural features of the network under the condition of no external feature supplementary modeling.

Trustworthy multi-view clustering via alternating generative adversarial representation learning and fusion

Multi-view clustering (MVC) has attached extensive attention as it provides an effective approach to deal with the unlabeled data in real-world applications. To enjoy the strong feature extraction capacity of deep learning, traditional shallow MVC methods are further extended to deep version. Though achieve superiorities in many fields, most existing deep MVC models are still limited by their lacking assurance of the trustworthiness and robustness. This drawback also leads to the possible poor clustering performance. To tackle this drawback, we propose an alternating generative adversarial learning strategy based network, termed as Alternating Generative Adversarial Representation Learning (AGARL) network for multi-view clustering, aiming to drive different views to fall into the same semantic space for the further alignment and information fusion. Various from the traditional generative adversarial learning strategy, the proposed alternating adversarial representation learning strategy successes in further aligning the distributions of the view-specific representations and exploring the cross-view consistency credited by its setting each view as the reference alternately. Then a consensus representative latent representation can be learned through an attention-like sub-network for all views. Finally, a self-supervised clustering module is employed in the proposed network to guide the learning of cluster assignment. In addition, comprehensive experiments on numerous benchmark MVC datasets have been performed to demonstrate the effectiveness of the proposed network compared to other state-of-the-art methods.