Network Representation Learning Research Articles

Identification of dynamic networks community by fusing deep learning and evolutionary clustering

Community detection is a critical component of network analysis and a hot topic in social computing. Detecting community structure in dynamic networks has important theoretical and practical implications for understanding the intrinsic function of networks and predicting network behavior. However, the majority of existing dynamic community detection methods adopt shallow models, which have limited ability to excavate complex non-linear structures and tend to generate undesirable community structures. In order to obtain an accurate and robust community structure in dynamic networks, we are inspired by network representation learning and utilize the deep learning to detect evolving communities in dynamic networks. In this paper, we propose a novel dynamic community detection method by fusing Deep Learning and Evolutionary Clustering (DLEC). This work attempts to combine deep learning and evolutionary clustering into a unified framework. First, we propose a matrix construction strategy to fully reveal the inherent community structures via the underlying community memberships. Then, we develop a novel multi-layer deep autoencoder framework that consists of multiple non-linear functions to extract the latent deep representation of the dynamic network. Based on the evolutionary clustering framework, a graph regularization term is introduced to ensure the smoothness of the community evolution. Finally, we employ the K-means clustering algorithm on the low-dimensional network space to obtain the community structure. Extensive experimental results on synthetic and real-world networks show that the proposed DLEC algorithm can effectively detect high-quality communities in dynamic networks.

Efficient degradation representation learning network for remote sensing image super-resolution

The advancements in convolutional neural networks have led to significant progress in image super-resolution (SR) techniques. Nevertheless, it is crucial to acknowledge that current SR methods operate under the assumption of bicubic downsampling as a degradation factor in low-resolution (LR) images and train models accordingly. However, this approach does not account for the unknown degradation patterns present in real-world scenes. To address this problem, we propose an efficient degradation representation learning network (EDRLN). Specifically, we adopt a contrast learning approach, which enables the model to distinguish and learn various degradation representations in realistic images to obtain critical degradation information. We also introduce streamlined and efficient pixel attention to strengthen the feature extraction capability of the model. In addition, we optimize our model with mutual affine convolution layers instead of ordinary convolution layers to make it more lightweight while minimizing performance loss. Experimental results on remote sensing and benchmark datasets show that our proposed EDRLN exhibits good performance for different degradation scenarios, while the lightweight version minimizes the performance loss as much as possible. The Code will be available at: https://github.com/Leilei11111/EDRLN.

Multi-view learning framework for predicting unknown types of cancer markers via directed graph neural networks fitting regulatory networks.

The discovery of diagnostic and therapeutic biomarkers for complex diseases, especially cancer, has always been a central and long-term challenge in molecular association prediction research, offering promising avenues for advancing the understanding of complex diseases. To this end, researchers have developed various network-based prediction techniques targeting specific molecular associations. However, limitations imposed by reductionism and network representation learning have led existing studies to narrowly focus on high prediction efficiency within single association type, thereby glossing over the discovery of unknown types of associations. Additionally, effectively utilizing network structure to fit the interaction properties of regulatory networks and combining specific case biomarker validations remains an unresolved issue in cancer biomarker prediction methods. To overcome these limitations, we propose a multi-view learning framework, CeRVE, based on directed graph neural networks (DGNN) for predicting unknown type cancer biomarkers. CeRVE effectively extracts and integrates subgraph information through multi-view feature learning. Subsequently, CeRVE utilizes DGNN to simulate the entire regulatory network, propagating node attribute features and extracting various interaction relationships between molecules. Furthermore, CeRVE constructed a comparative analysis matrix of three cancers and adjacent normal tissues through The Cancer Genome Atlas and identified multiple types of potential cancer biomarkers through differential expression analysis of mRNA, microRNA, and long noncoding RNA. Computational testing of multiple types of biomarkers for 72 cancers demonstrates that CeRVE exhibits superior performance in cancer biomarker prediction, providing a powerful tool and insightful approach for AI-assisted disease biomarker discovery.

DRADTiP: Drug repurposing for aging disease through drug-target interaction prediction

MotivationThe greatest risk factor for many non-communicable diseases is aging. Studies on model organisms have demonstrated that genetic and chemical perturbation alterations can lengthen longevity and overall health. However, finding longevity-enhancing medications and their related targets is difficult. MethodIn this work, we designed a novel drug repurposing model by identifying the interaction between aging-related genes or targets and drugs similar to aging disease. Each disease is associated with certain specific genetic factors for the occurrence of that disease. The factors include gene expression, pathway, miRNA, and degree of genes in the protein-protein interaction network. In this paper, we aim to find the drugs that prolong the life span of humans with their aging-related targets using the above-mentioned factors. In addition, the contribution or importance of each factor may vary among drugs and targets. Therefore, we designed a novel multi-layer random walk-based network representation learning model including node and edge weight to learn the features of drugs and targets respectively. ResultThe performance of the proposed model is demonstrated using k-fold cross-validation (k = 5). This model achieved better performance with scores of 0.93 and 0.91 for precision and recall respectively. The drugs identified by the system are evaluated to be potential candidates for aging since the degree of interaction between the potential drugs and their gene sets are high. In addition, the genes that are interacting with drugs produce the same biological functions. Hence the life span of the human will be increased or prolonged.

SiGNN: A spike-induced graph neural network for dynamic graph representation learning

In the domain of dynamic graph representation learning (DGRL), capturing the temporal evolution within real-world networks is of paramount importance. Spiking Neural Networks (SNNs), renowned for their temporal dynamics and low-power characteristics, provide an efficient solution for temporal processing in DGRL tasks. However, due to the spike-based information encoding mechanism of SNNs, existing DGRL methods that employ SNNs face significant limitations in their representational capacity. To address this issue, we propose an innovative framework named Spike-induced Graph Neural Network (SiGNN) for learning enhanced spatiotemporal representations on dynamic graphs. Specifically, we achieve a harmonious integration of SNNs and GNNs through a novel Temporal Activation (TA) mechanism. This mechanism enables SiGNN to effectively harness the temporal dynamics of SNNs while circumventing the representational constraints imposed by the binary nature of spikes. Furthermore, leveraging the inherent adaptability of SNNs, SiGNN incorporates an in-depth analysis of evolutionary patterns within graphs across multiple time granularities, thereby facilitating the acquisition of multiscale temporal node representations. Extensive experiments on various real-world dynamic graph datasets demonstrate the superior performance of SiGNN in the node classification task. Our code is publicly available at https://github.com/CharliedoD/SiGNN.

Collaborative graph neural networks for augmented graphs: A local-to-global perspective

In the field of graph neural networks (GNNs) for representation learning, a noteworthy highlight is the potential of embedding fusion architectures for augmented graphs. However, prevalent GNN embedding fusion architectures mainly focus on handling graph combinations from a global perspective, often ignoring their collaboration with the information of local graph combinations. This inherent limitation constrains the ability of the constructed models to handle multiple input graphs, particularly when dealing with noisy input graphs collected from error-prone sources or those resulting from deficiencies in graph augmentation methods. In this paper, we propose an effective and robust embedding fusion architecture from a local-to-global perspective termed collaborative graph neural networks for augmented graphs (LoGo-GNN). Essentially, LoGo-GNN leverages a pairwise graph combination scheme to generate local perspective inputs. Together with the global graph combination, this serves as the basis to generate a local-to-global perspective. Specifically, LoGo-GNN employs a perturbation augmentation strategy to generate multiple augmentation graphs, thereby facilitating collaboration and embedding fusion from a local-to-global perspective through the use of graph combinations. In addition, LoGo-GNN incorporates a novel loss function for learning complementary information between different perspectives. We also conduct theoretical analysis to assess its expressive power under ideal conditions, demonstrating the effectiveness of LoGo-GNN. Our experiments, focusing on node classification and clustering tasks, highlight the superior performance of LoGo-GNN compared to state-of-the-art methods. Additionally, robustness analysis further confirms its effectiveness in addressing uncertainty challenges.

Temporal Protein Complex Identification Based on Dynamic Heterogeneous Protein Information Network Representation Learning.

Protein complexes, as the fundamental units of cellular function and regulation, play a crucial role in understanding the normal physiological functions of cells. Existing methods for protein complex identification attempt to introduce other biological information on top of the protein-protein interaction (PPI) network to assist in evaluating the degree of association between proteins. However, these methods usually treat protein interaction networks as flat homogeneous static networks. They cannot distinguish the roles and importance of different types of biological information, nor can they reflect the dynamic changes of protein complexes. In recent years, heterogeneous network representation learning has achieved great success in processing complex heterogeneous information and mining deep semantics. We thus propose a temporal protein complex identification method based on Dynamic Heterogeneous Protein information network Representation Learning, DHPRL. DHPRL naturally integrates multiple types of heterogeneous biological information in the cellular temporal dimension. It simultaneously models the temporal dynamic properties of proteins and the heterogeneity of biological information to improve the understanding of protein interactions and the accuracy of complex prediction. Firstly, we construct Dynamic Heterogeneous Protein Information Network (DHPIN) by integrating temporal gene expression information and GO attribute information. Then we design a dual-view collaborative contrast mechanism. Specifically, proposing to learn protein representations from two views of DHPIN (1-hop relation view and meta-path view) to model the consistency and specificity between nearest-neighbour bio information and deeper biological semantics. The dynamic PPI network is thereafter re-weighted based on the learned protein representations. Finally, we perform protein identification on the re-weighted dynamic PPI network. Extensive experimental results demonstrate that DHPRL can effectively model complicated biological information and achieve state-of-the-art performance in most cases.

Fuzzy Neural Network for Representation Learning on Uncertain Graphs

Fuzzy Neural Network for Representation Learning on Uncertain Graphs

FluPMT: Prediction of Predominant Strains of Influenza A Viruses via Multi-Task Learning.

Seasonal influenza vaccines play a crucial role in saving numerous lives annually. However, the constant evolution of the influenza A virus necessitates frequent vaccine updates to ensure its ongoing effectiveness. The decision to develop a new vaccine strain is generally based on the assessment of the current predominant strains. Nevertheless, the process of vaccine production and distribution is very time-consuming, leaving a window for the emergence of new variants that could decrease vaccine effectiveness, so predictions of influenza A virus evolution can inform vaccine evaluation and selection. Hence, we present FluPMT, a novel sequence prediction model that applies an encoder-decoder architecture to predict the hemagglutinin (HA) protein sequence of the upcoming season's predominant strain by capturing the patterns of evolution of influenza A viruses. Specifically, we employ time series to model the evolution of influenza A viruses, and utilize attention mechanisms to explore dependencies among residues of sequences. Additionally, antigenic distance prediction based on graph network representation learning is incorporated into the sequence prediction as an auxiliary task through a multi-task learning framework. Experimental results on two influenza datasets highlight the exceptional predictive performance of FluPMT, offering valuable insights into virus evolutionary dynamics, as well as vaccine evaluation and production.

Language-guided temporal primitive modeling for skeleton-based action recognition

Human actions describe the complex body dynamics, which are defined by an ordered set of sub-action primitives. Existing methods in skeleton-based action recognition primarily focus on designing various networks to learn the entire action representation, in alignment with the action label. However, training the heterogeneous action as a whole increases the burden on the network and is not conducive to learning the temporal structure of the action. In this paper, the traditional action label is extended into the ordered primitive language description by employing a large language model, which not only unveils the temporal structure of the action but also enriches its connotation. Based on this, we propose a language-guided skeleton representation learning network (LGS-Net) that leverages the language description to guide the skeleton feature learning from the ordered primitive and global action levels. Especially, the primitive guidance aligns the features of skeleton clips with those of the ordered primitive descriptions while preserving temporal order, which promotes the network to model the temporal primitive and capture the internal structure within the action from skeletons. To enhance the cross-modal alignment performance, we develop an innovative temporal alignment loss supplemented with diversity and sparsity regularization terms to generate the discriminative multi-modal representation. Evaluated on three benchmark datasets, NTU-60, NTU-120 and N-UCLA, the proposed LGS-Net achieves comparable results to state-of-the-art methods, which proves the effectiveness of the language-guided learning mechanism.

SiSRS: Signed social recommender system using deep neural network representation learning

Most approaches used in social recommender systems concentrate mostly on positive relationshipswithin the social network, frequently ignoring the insightful information that negative relationships can offer. A more thorough understanding of user characteristics and behaviors within a social network can result from integrating positive and negative links to construct a signed graph. However, there are two major challenges in developing signed social recommender systems. Considering negative links in these systems needs to be done carefully to make sure that these relationships are appropriately represented and used. Moreover, it is difficult to predict user interactions and preferences effectively when relationships represented by positive and negative links conflict, known as social inconsistency. Therefore, working with signed graphs requires the management of social inconsistency. To address these issues, in this paper, a signed social recommender system called SiSRS is presented using a deep architecture combined with network representation learning. The SiSRS is composed of three principal components. First, the application of signed graph attention networks is explored to collate and disseminate information across the network via graph motifs, leading to the creation of node embeddings. Second, a deep autoencoder model is employed to assimilate top-k semantic signed social data within the deep learning framework. Third, a novel loss function is formulated to reduce the impact of social inconsistency. The efficacy of the SiSRS model was evaluated through extensive experiments conducted on two datasets in terms of various evaluation metrics. The results indicated a superior performance of this approach compared to existing state-of-the-art methods.

Deciphering the Language of Protein-DNA Interactions: A Deep Learning Approach Combining Contextual Embeddings and Multi-Scale Sequence Modeling

Deciphering the mechanisms governing protein-DNA interactions is crucial for understanding key cellular processes and disease pathways. In this work, we present a powerful deep learning approach that significantly advances the computational prediction of DNA-interacting residues from protein sequences.Our method leverages the rich contextual representations learned by pre-trained protein language models, such as ProtTrans, to capture intrinsic biochemical properties and sequence motifs indicative of DNA binding sites. We then integrate these contextual embeddings with a multi-window convolutional neural network architecture, which scans across the sequence at varying window sizes to effectively identify both local and global binding patterns.Comprehensive evaluation on curated benchmark datasets demonstrates the remarkable performance of our approach, achieving an area under the ROC curve (AUC) of 0.89 – a substantial improvement over previous state-of-the-art sequence-based predictors. This showcases the immense potential of pairing advanced representation learning and deep neural network designs for uncovering the complex syntax governing protein-DNA interactions directly from primary sequences.Our work not only provides a robust computational tool for characterizing DNA-binding mechanisms, but also highlights the transformative opportunities at the intersection of language modeling, deep learning, and protein sequence analysis. The publicly available code and data further facilitate broader adoption and continued development of these techniques for accelerating mechanistic insights into vital biological processes and disease pathways.In addition, the code and data for this work are available at https://github.com/B1607/DIRP.

Predicting transcription factor binding sites by a multi-modal representation learning method based on cross-attention network

The prediction of transcription factor binding sites (TFBS) plays a crucial role in studying cellular functions and understanding transcriptional regulatory processes. With the development of chromatin immunoprecipitation sequencing (ChIP-seq) technology, an increasing number of computer-aided TFBS prediction models have emerged. However, how to integrate multi-modal information of DNA and obtain efficient features to improve prediction accuracy remains a major challenge. Here, we propose MultiTF, a multi-modal representation learning method based on a cross-attention network for predicting transcription factor binding sites. Among TFBS prediction methods, we are the first to use graph neural networks and cross-attention networks for representation learning. MultiTF uses dna2vec to extract global contextual features of DNA sequences, DNAshapeR to extract shape features, and the CDPfold model and graph attention network for learning and representation of DNA structural features. Finally, with the help of our cross-attention module, we successfully combine sequence, structural, and shape features to achieve interactive fusion. When comparing MultiTF to other state-of-the-art methods using 165 ENCODE ChIP-seq datasets, we find that MultiTF exhibits average ACC, ROC-AUC, and PR-AUC values of 0.911, 0.978, and 0.982, respectively. The results show that MultiTF achieves unprecedented prediction accuracy compared to previous TFBS prediction models. In addition, our visual analysis of structural features provides interpretability for the prediction results.

Multi-dimensional feature fusion-based expert recommendation in community question answering

PurposeCommunity question answering (CQA) platforms play a significant role in knowledge dissemination and information retrieval. Expert recommendation can assist users by helping them find valuable answers efficiently. Existing works mainly use content and user behavioural features for expert recommendation, and fail to effectively leverage the correlation across multi-dimensional features.Design/methodology/approachTo address the above issue, this work proposes a multi-dimensional feature fusion-based method for expert recommendation, aiming to integrate features of question–answerer pairs from three dimensions, including network features, content features and user behaviour features. Specifically, network features are extracted by first learning user and tag representations using network representation learning methods and then calculating questioner–answerer similarities and answerer–tag similarities. Secondly, content features are extracted from textual contents of questions and answerer generated contents using text representation models. Thirdly, user behaviour features are extracted from user actions observed in CQA platforms, such as following and likes. Finally, given a question–answerer pair, the three dimensional features are fused and used to predict the probability of the candidate expert answering the given question.FindingsThe proposed method is evaluated on a data set collected from a publicly available CQA platform. Results show that the proposed method is effective compared with baseline methods. Ablation study shows that network features is the most important dimensional features among all three dimensional features.Practical implicationsThis work identifies three dimensional features for expert recommendation in CQA platforms and conducts a comprehensive investigation into the importance of features for the performance of expert recommendation. The results suggest that network features are the most important features among three-dimensional features, which indicates that the performance of expert recommendation in CQA platforms is likely to get improved by further mining network features using advanced techniques, such as graph neural networks. One broader implication is that it is always important to include multi-dimensional features for expert recommendation and conduct systematic investigation to identify the most important features for finding directions for improvement.Originality/valueThis work proposes three-dimensional features given that existing works mostly focus on one or two-dimensional features and demonstrate the effectiveness of the newly proposed features.

Neighbor-Enhanced Representation Learning for Link Prediction in Dynamic Heterogeneous Attributed Networks

Dynamic link prediction aims to predict future connections among unconnected nodes in a network. It can be applied for friend recommendations, link completion, and other tasks. Network representation learning algorithms have demonstrated considerable effectiveness in various prediction tasks. However, most network representation learning algorithms are based on homogeneous networks and static networks for link prediction that do not consider rich semantic and dynamic information. Additionally, existing dynamic network representation learning methods neglect the neighborhood interaction structure of the node. In this work, we design a neighbor-enhanced dynamic heterogeneous attributed network embedding method (NeiDyHNE) for link prediction. In light of the impressive achievements of the heuristic methods, we learn the information of common neighbors and neighbors’ interaction in heterogeneous networks to preserve the neighbors proximity and common neighbors proximity. NeiDyHNE encodes the attributes and neighborhood structure of nodes as well as the evolutionary features of the dynamic network. More specifically, NeiDyHNE consists of the hierarchical structure attention module and the convolutional temporal attention module. The hierarchical structure attention module captures the rich features and semantic structure of nodes. The convolutional temporal attention module captures the evolutionary features of the network over time in dynamic heterogeneous networks. We evaluate our method and various baseline methods on the dynamic link prediction task. Experimental results demonstrate that our method is superior to baseline methods in terms of accuracy.

Explainability of deep reinforcement learning algorithms in robotic domains by using Layer-wise Relevance Propagation

A key component to the recent success of reinforcement learning is the introduction of neural networks for representation learning. Doing so allows for solving challenging problems in several domains, one of which is robotics. However, a major criticism of deep reinforcement learning (DRL) algorithms is their lack of explainability and interpretability. This problem is even exacerbated in robotics as they oftentimes cohabitate space with humans, making it imperative to be able to reason about their behavior. In this paper, we propose to analyze the learned representation in a robotic setting by utilizing Graph Networks (GNs). Using the GN and Layer-wise Relevance Propagation (LRP), we represent the observations as an entity-relationship to allow us to interpret the learned policy. We evaluate our approach in two environments in MuJoCo. These two environments were delicately designed to effectively measure the value of knowledge gained by our approach to analyzing learned representations. This approach allows us to analyze not only how different parts of the observation space contribute to the decision-making process but also differentiate between policies and their differences in performance. This difference in performance also allows for reasoning about the agent’s recovery from faults. These insights are key contributions to explainable deep reinforcement learning in robotic settings.

Multi-graph aggregated graph neural network for heterogeneous graph representation learning

Multi-graph aggregated graph neural network for heterogeneous graph representation learning

Social user role value analysis and trusted user autonomous diffusion for participatory crowdsensing

Social participatory-sensing, as an emerging research field, is extremely challenging to recruit trustworthy and active high-value users in a sparse user context. And the richness and closeness of users' social relationships provide new research ideas. Therefore, a participatory sensing user recruitment method based on dynamic social network role detection (DSRD) is proposed, in which firstly, multiple identity information of users is mined based on the decomposition of their overlapping social relationships to filter out high-quality sensing users. Secondly, based on role-oriented network representation learning, it models users' role information and establishes a role hierarchy model to evaluate users' social functions and role values. Finally, the concept of temporal social centrality is proposed for the first time for integrating users' social and network structural features to assess the overall value of users and ensure the coverage of task assignments under a sparse user pool. Experimental results on the open datasets Gowalla and Brightkite show that under the constraints of cost budget and number of users, the proposed user recruitment framework DSRD effectively improves task coverage with less time overhead compared to the baseline algorithm.

Research on the dynamic spread of information in social networks based on relationship strength theory and feedback mechanism

Introduction: Studying the main factors and related paths of rumor propagation contributes to the precise governance of rumor information in social networks. Most existing network representation learning methods do not fit with real-world information propagation networks well, and the network cannot effectively model the temporal characteristics and dynamic evolution features of rumor information propagation.Methods: Our study proposes a new dynamic network representation model for information propagation. Additionally, the study introduces a feedback mechanism where the latest node representations are fed back to neighboring nodes.Results: The method solves the problem of delayed network representation and improves network representation performance.Discussion: We conducted experimental simulations, and the results indicate that a higher level of trust contributes to stable group relationships and converging opinions, reducing the likelihood of opinion dispersion. Furthermore, novelty of topics, and interactivity between users, and opinion leaders exhibit distinct characteristics in guiding public opinion. The viewpoint evolution of the newly constructed dynamic network representation model aligns more closely with viewpoint evolution in real-world social networks.

T-distributed Stochastic Neighbor Network for unsupervised representation learning

Unsupervised representation learning (URL) is still lack of a reasonable operator (e.g. convolution kernel) for exploring meaningful structural information from generic data including vector, image and tabular data. In this paper, we propose a simple end-to-end T-distributed Stochastic Neighbor Network (TsNet) for URL with clustering downstream task. Concretely, our TsNet model has three major components: (1) an adaptive connectivity distribution learning module is presented to construct a pairwise graph for preserving the local structure of generic data; (2) a T-distributed stochastic neighbor embedding based loss function is designed to learn a transformation between embeddings and original data, which improves the discrimination of representations; (3) a nonlinear parametric mapping is learned via our TsNet on an unsupervised generalized manner, which can address the “out-of-sample” issue. By combining these components, our method is able to considerably outperform previous related unsupervised learning approaches on visualization and clustering of generic data. A simple deep neural network equipped on our model respectively achieves 74.90%, 76.56% ACC and NMI, which is 8% relative improvement over previous state-of-the-art on real single-cell RNA-sequencing (scRNA-seq) datasets clustering.