Network Representation Learning Algorithms Research Articles

Neighbor-Enhanced Representation Learning for Link Prediction in Dynamic Heterogeneous Attributed Networks

Dynamic link prediction aims to predict future connections among unconnected nodes in a network. It can be applied for friend recommendations, link completion, and other tasks. Network representation learning algorithms have demonstrated considerable effectiveness in various prediction tasks. However, most network representation learning algorithms are based on homogeneous networks and static networks for link prediction that do not consider rich semantic and dynamic information. Additionally, existing dynamic network representation learning methods neglect the neighborhood interaction structure of the node. In this work, we design a neighbor-enhanced dynamic heterogeneous attributed network embedding method (NeiDyHNE) for link prediction. In light of the impressive achievements of the heuristic methods, we learn the information of common neighbors and neighbors’ interaction in heterogeneous networks to preserve the neighbors proximity and common neighbors proximity. NeiDyHNE encodes the attributes and neighborhood structure of nodes as well as the evolutionary features of the dynamic network. More specifically, NeiDyHNE consists of the hierarchical structure attention module and the convolutional temporal attention module. The hierarchical structure attention module captures the rich features and semantic structure of nodes. The convolutional temporal attention module captures the evolutionary features of the network over time in dynamic heterogeneous networks. We evaluate our method and various baseline methods on the dynamic link prediction task. Experimental results demonstrate that our method is superior to baseline methods in terms of accuracy.

Multi-channel high-order network representation learning research.

The existing network representation learning algorithms mainly model the relationship between network nodes based on the structural features of the network, or use text features, hierarchical features and other external attributes to realize the network joint representation learning. Capturing global features of the network allows the obtained node vectors to retain more comprehensive feature information during training, thereby enhancing the quality of embeddings. In order to preserve the global structural features of the network in the training results, we employed a multi-channel learning approach to perform high-order feature modeling on the network. We proposed a novel algorithm for multi-channel high-order network representation learning, referred to as the Multi-Channel High-Order Network Representation (MHNR) algorithm. This algorithm initially constructs high-order network features from the original network structure, thereby transforming the single-channel network representation learning process into a multi-channel high-order network representation learning process. Then, for each single-channel network representation learning process, the novel graph assimilation mechanism is introduced in the algorithm, so as to realize the high-order network structure modeling mechanism in the single-channel network representation learning. Finally, the algorithm integrates the multi-channel and single-channel mechanism of high-order network structure joint modeling, realizing the efficient use of network structure features and sufficient modeling. Experimental results show that the node classification performance of the proposed MHNR algorithm reaches a good order on Citeseer, Cora, and DBLP data, and its node classification performance is better than that of the comparison algorithm used in this paper. In addition, when the vector length is optimized, the average classification accuracy of nodes of the proposed algorithm is up to 12.24% higher than that of the DeepWalk algorithm. Therefore, the node classification performance of the proposed algorithm can reach the current optimal order only based on the structural features of the network under the condition of no external feature supplementary modeling.

A Network Representation Learning Model Based on Multiple Remodeling of Node Attributes

Current network representation learning models mainly use matrix factorization-based and neural network-based approaches, and most models still focus only on local neighbor features of nodes. Knowledge representation learning aims to learn low-dimensional dense representations of entities and relations from structured knowledge graphs, and most models use the triplets to capture semantic, logical, and topological features between entities and relations. In order to extend the generalization capability of the network representation learning models, this paper proposes a network representation learning algorithm based on multiple remodeling of node attributes named MRNR. The model constructs the knowledge triplets through the textual association relationships between nodes. Meanwhile, a novel co-occurrence word training method has been proposed. Multiple remodeling of node attributes can significantly improve the effectiveness of network representation learning. At the same time, MRNR introduces the attention mechanism to achieve the weight information for key co-occurrence words and triplets, which further models the semantic and topological features between entities and relations, and it makes the network embedding more accurate and has better generalization ability.

AMGDTI: drug-target interaction prediction based on adaptive meta-graph learning in heterogeneous network.

Prediction of drug-target interactions (DTIs) is essential in medicine field, since it benefits the identification of molecular structures potentially interacting with drugs and facilitates the discovery and reposition of drugs. Recently, much attention has been attracted to network representation learning to learn rich information from heterogeneous data. Although network representation learning algorithms have achieved success in predicting DTI, several manually designed meta-graphs limit the capability of extracting complex semantic information. To address the problem, we introduce an adaptive meta-graph-based method, termed AMGDTI, for DTI prediction. In the proposed AMGDTI, the semantic information is automatically aggregated from a heterogeneous network by training an adaptive meta-graph, thereby achieving efficient information integration without requiring domain knowledge. The effectiveness of the proposed AMGDTI is verified on two benchmark datasets. Experimental results demonstrate that the AMGDTI method overall outperforms eight state-of-the-art methods in predicting DTI and achieves the accurate identification of novel DTIs. It is also verified that the adaptive meta-graph exhibits flexibility and effectively captures complex fine-grained semantic information, enabling the learning of intricate heterogeneous network topology and the inference of potential drug-target relationship.

APTM: Structurally Informative Network Representation Learning

Network representation learning algorithms provide a method to map complex network data into low-dimensional real vectors, aiming to capture and preserve structural information within the network. In recent years, these algorithms have found widespread applications in tasks such as link prediction and node classification in graph data mining. In this work, we propose a novel algorithm based on an adaptive transfer probability matrix. We use a deep neural network, comprising an autoencoder, to encode and reduce the dimensionality of the generated matrix, thereby encoding the intricate structural information of the network into low-dimensional real vectors. We evaluate the algorithm's performance through node classification, and in comparison with mainstream network representation learning algorithms, our proposed algorithm demonstrates favorable results. It outperforms baseline models in terms of micro-F1 scores on three datasets: PPI, Citeseer, and Wiki.

MultiSimNeNc: A network representation learning-based module identification method by network embedding and clustering

Accurate identification of gene modules based on biological networks is an effective approach to understanding gene patterns of cancer from a module-level perspective. However, most graph clustering algorithms just consider low-order topological connectivity, which limits their accuracy in gene module identification. In this study, we propose a novel network-based method, MultiSimNeNc, to identify modules in various types of networks by integrating network representation learning (NRL) and clustering algorithms. In this method, we first obtain the multi-order similarity of the network using graph convolution (GC). Then, we aggregate the multi-order similarity to characterize the network structure and use non-negative matrix factorization (NMF) to achieve low-dimensional node characterization. Finally, we predict the number of modules based on the bayesian information criterion (BIC) and use the gaussian mixture model (GMM) to identify modules. To testify to the efficacy of MultiSimeNc in module identification, we apply this method to two types of biological networks and six benchmark networks, where the biological networks are constructed based on the fusion of multi-omics data from glioblastoma (GBM). The analysis shows that MultiSimNeNc outperforms several state-of-the-art module identification algorithms in identification accuracy, which is an effective method for understanding biomolecular mechanisms of pathogenesis from a module-level perspective.

Prediction of potential small molecule-miRNA associations based on heterogeneous network representation learning.

MicroRNAs (miRNAs) are closely associated with the occurrences and developments of many complex human diseases. Increasing studies have shown that miRNAs emerge as new therapeutic targets of small molecule (SM) drugs. Since traditional experiment methods are expensive and time consuming, it is particularly crucial to find efficient computational approaches to predict potential small molecule-miRNA (SM-miRNA) associations. Considering that integrating multi-source heterogeneous information related with SM-miRNA association prediction would provide a comprehensive insight into the features of both SMs and miRNAs, we proposed a novel model of Small Molecule-MiRNA Association prediction based on Heterogeneous Network Representation Learning (SMMA-HNRL) for more precisely predicting the potential SM-miRNA associations. In SMMA-HNRL, a novel heterogeneous information network was constructed with SM nodes, miRNA nodes and disease nodes. To access and utilize of the topological information of the heterogeneous information network, feature vectors of SM and miRNA nodes were obtained by two different heterogeneous network representation learning algorithms (HeGAN and HIN2Vec) respectively and merged with connect operation. Finally, LightGBM was chosen as the classifier of SMMA-HNRL for predicting potential SM-miRNA associations. The 10-fold cross validations were conducted to evaluate the prediction performance of SMMA-HNRL, it achieved an area under of ROC curve of 0.9875, which was superior to other three state-of-the-art models. With two independent validation datasets, the test experiment results revealed the robustness of our model. Moreover, three case studies were performed. As a result, 35, 37, and 22 miRNAs among the top 50 predicting miRNAs associated with 5-FU, cisplatin, and imatinib were validated by experimental literature works respectively, which confirmed the effectiveness of SMMA-HNRL. The source code and experimental data of SMMA-HNRL are available at https://github.com/SMMA-HNRL/SMMA-HNRL.

Web Services Clustering via Exploring Unified Content and Structural Semantic Representation

Clustering Web services can improve the quality and efficiency of service discovery and management within a service repository. Nowadays, Web services frequently interact (e.g., composition relation and tag sharing relation) with each other to form a complex and heterogeneous service relationship network. The rich network relations inherently reflect either positive or negative clustering association between Web services, which can be a strong supplement to service semantics for characterizing functional affinities between Web services. In this paper, we propose to cluster Web services by utilizing both description documents and the structural information from the service relationship network. We first learn the content semantic information from service description documents based on the widely used Doc2vec model, and meanwhile, learn the structural semantic information from the service relationship network based on a network representation learning algorithm. Then, we propose to pretrain the content and structural semantic information to obtain the most relevant and unified features through training a service classification model with partially labeled data. Finally, a spectral clustering algorithm is utilized for Web services clustering based on the above unified features with preserved content and structural semantics. Therefore, the proposed services clustering approach takes advantage of both service content semantic and service network structure semantic based similarity between services. Extensive experiments are conducted on a real-world dataset from ProgrammableWeb, composed of 12919 Web API services. Experimental results demonstrate that our approach yields an improvement of 4.78% in precision and 5.4% in recall over the state-of-the-art method.

Attribute Network Representation Learning with Dual Autoencoders

The purpose of attribute network representation learning is to learn the low-dimensional dense vector representation of nodes by combining structure and attribute information. The current network representation learning methods have insufficient interaction with structure when learning attribute information, and the structure and attribute information cannot be well integrated. In this paper, we propose an attribute network representation learning method for dual-channel autoencoder. One channel is for the network structure, and adopting the multi-hop attention mechanism is used to capture the node’s high-order neighborhood information and calculate the neighborhood weight; The other channel is for the node attribute information, and a low-pass Laplace filter is designed to iteratively obtain the attribute information in the neighborhood of the node. The dual-channel autoencoder ensures the learning of structure and attribute information respectively. The adaptive fusion module is constructed in this method to increase the acquisition of important information through the consistency and difference constraints of two kinds of information. The method trains encoders by supervising the joint reconstruction of loss functions of two autoencoders. Based on the node clustering task on four authentic open data sets, and compared with eight network representation learning algorithms in clustering accuracy, standardized mutual information and running time of some algorithms, the experimental results show that the proposed method is superior and reasonable.

Learning Behavior Evaluation Model and Teaching Strategy Innovation by Social Media Network Following Learning Psychology.

With the development of various network technologies and the spread of coronavirus disease 2019, many online learning platforms have been built. However, some of them may negatively impact student learning outcomes. Therefore, this study aims to improve the online learning effect of students by comprehensively evaluating their learning behavior by using deep learning algorithms. On this basis, new teaching strategies are proposed. According to the structured deep network embedding model, a network representation learning algorithm is proposed with the help of auto-encoders under deep learning. This study elaborates the concept and structure of the encoder model and tests its performance. After the node labels and dataset are trained, the applicable parameter λ2 of the model is 0.3. During the teaching process, the model’s reliability in distinguishing users is examined. Therefore, this model can be applied to network teaching, is an innovative teaching strategy, and provides a theoretical basis for improving teaching methods.

Virtual Power Plant Partition Strategy Based on Network Representation

The virtual power plant (VPP) is an important technology for realizing smart grids. VPPs are not limited by geographical location, which greatly aggregates the distributed energy resources. To divide a VPP across regions, we no longer limit it to power lines. This paper proposes a three-stage strategy for the classification of VPPs that includes graph model construction, communication capacity assessment and cross-regional classification. To construct the VPP graph model, complex network theory and communication matrices are used. In addition, a network representation learning and clustering algorithm is used to classify the VPP decision areas. Then, the classification results are evaluated using modularity and network communication sequence entropy to identify VPP boundaries. Finally, the validity of the method is verified by simulating the test network and the real network.

Distinguishing Homophily from Peer Influence Through Network Representation Learning

Peer influence and homophily are two entangled forces underlying social influences. However, distinguishing homophily from peer influence is difficult, particularly when there is latent homophily caused by unobservable features. This paper proposes a novel data-driven framework that combines the advantages of latent homophily identification and causal inference. Specifically, the approach first utilizes scalable network representation learning algorithms to obtain node embeddings, which are extracted from social network structures. Then, the embeddings are used to control latent homophily in a quasi-experimental design for causal inference. The simulation experiments show that the proposed approach can estimate peer influence more accurately than existing parameterized approaches and data-driven methods. We applied the proposed framework in an empirical study of players’ online gaming behaviors. First, our approach can achieve improved model fitness for estimating peer influence in online games. Second, we discover a heterogeneous effect of peer influence: players with higher tenure and playing levels receive stronger peer influence. Finally, our results suggest that the homophily effect has a stronger influence on players’ behavior than peer influence. Summary of Contribution: The study proposes a novel computational method to separate peer influence from homophily in an online network. Using network embeddings learned from data to control latent homophily, the approach effectively addresses the challenge of correctly identifying peer effects in the absence of randomized experimental conditions. While simplifying the computational process, the method achieves good computational performance, thus effectively helping researchers and practitioners extract useful network information in various online service contexts.

Network Representation Learning Algorithm Based on Community Folding

<p>Network representation learning is a machine learning method that maps network topology and node information into low-dimensional vector space, which can reduce the temporal and spatial complexity of downstream network data mining such as node classification and graph clustering. This paper addresses the problem that neighborhood information-based network representation learning algorithm ignores the global topological information of the network. We propose the Network Representation Learning Algorithm Based on Community Folding (CF-NRL) considering the influence of community structure on the global topology of the network. Each community of the target network is regarded as a folding unit, the same network representation learning algorithm is used to learn the vector representation of the nodes on the folding network and the target network, then the vector representations are spliced correspondingly to obtain the final vector representation of the node. Experimental results show the excellent performance of the proposed algorithm.</p> <p> </p>

Network Representation Learning Algorithm Based on Complete Subgraph Folding

Network representation learning is a machine learning method that maps network topology and node information into low-dimensional vector space. Network representation learning enables the reduction of temporal and spatial complexity in the downstream data mining of networks, such as node classification and graph clustering. Existing algorithms commonly ignore the global topological information of the network in network representation learning, leading to information loss. The complete subgraph in the network commonly has a community structure, or it is the component module of the community structure. We believe that the structure of the community serves as the revealed structure in the topology of the network and preserves global information. In this paper, we propose SF-NRL, a network representation learning algorithm based on complete subgraph folding. The algorithm preserves the global topological information of the original network completely, by finding complete subgraphs in the original network and folding them into the super nodes. We employ the network representation learning algorithm to study the node embeddings on the folded network, and then merge the embeddings of the folded network with those of the original network to obtain the final node embeddings. Experiments performed on four real-world networks prove the effectiveness of the SF-NRL algorithm. The proposed algorithm outperforms the baselines in evaluation metrics on community detection and multi-label classification tasks. The proposed algorithm can effectively generalize the global information of the network and provides excellent classification performance.

Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding

Network representation learning (NRL) advances the conventional graph mining of social networks, knowledge graphs, and complex biomedical and physics information networks. Dozens of NRL algorithms have been reported in the literature. Most of them focus on learning node embeddings for homogeneous networks, but they differ in the specific encoding schemes and specific types of node semantics captured and used for learning node embedding. This article reviews the design principles and the different node embedding techniques for NRL over homogeneous networks. To facilitate the comparison of different node embedding algorithms, we introduce a unified reference framework to divide and generalize the node embedding learning process on a given network into preprocessing steps, node feature extraction steps, and node embedding model training for an NRL task such as link prediction and node clustering. With this unifying reference framework, we highlight the representative methods, models, and techniques used at different stages of the node embedding model learning process. This survey not only helps researchers and practitioners gain an in-depth understanding of different NRL techniques but also provides practical guidelines for designing and developing the next generation of NRL algorithms and systems.

Inferring Users' Social Roles with a Multi-Level Graph Neural Network Model.

Users of social networks have a variety of social statuses and roles. For example, the users of Weibo include celebrities, government officials, and social organizations. At the same time, these users may be senior managers, middle managers, or workers in companies. Previous studies on this topic have mainly focused on using the categorical, textual and topological data of a social network to predict users’ social statuses and roles. However, this cannot fully reflect the overall characteristics of users’ social statuses and roles in a social network. In this paper, we consider what social network structures reflect users’ social statuses and roles since social networks are designed to connect people. Taking an Enron email dataset as an example, we analyzed a preprocessing mechanism used for social network datasets that can extract users’ dynamic behavior features. We further designed a novel social network representation learning algorithm in order to infer users’ social statuses and roles in social networks through the use of an attention and gate mechanism on users’ neighbors. The extensive experimental results gained from four publicly available datasets indicate that our solution achieves an average accuracy improvement of 2% compared with GraphSAGE-Mean, which is the best applicable inductive representation learning method.

Integrating Multimodal and Longitudinal Neuroimaging Data with Multi-Source Network Representation Learning.

Uncovering the complex network of the brain is of great interest to the field of neuroimaging. Mining from these rich datasets, scientists try to unveil the fundamental biological mechanisms in the human brain. However, neuroimaging data collected for constructing brain networks is generally costly, and thus extracting useful information from a limited sample size of brain networks is demanding. Currently, there are two common trends in neuroimaging data collection that could be exploited to gain more information: 1) multimodal data, and 2) longitudinal data. It has been shown that these two types of data provide complementary information. Nonetheless, it is challenging to learn brain network representations that can simultaneously capture network properties from multimodal as well as longitudinal datasets. Here we propose a general fusion framework for multi-source learning of brain networks - multimodal brain network fusion with longitudinal coupling (MMLC). In our framework, three layers of information are considered, including cross-sectional similarity, multimodal coupling, and longitudinal consistency. Specifically, we jointly factorize multimodal networks and construct a rotation-based constraint to couple network variance across time. We also adopt the consensus factorization as the group consistent pattern. Using two publicly available brain imaging datasets, we demonstrate that MMLC may better predict psychometric scores than some other state-of-the-art brain network representation learning algorithms. Additionally, the discovered significant brain regions are synergistic with previous literature. Our new approach may boost statistical power and sheds new light on neuroimaging network biomarkers for future psychometric prediction research by integrating longitudinal and multimodal neuroimaging data.

Interaction event network modeling based on temporal point process

Interaction event networks, which consist of interaction events among a set of individuals, exist in many areas from social, biological to financial applications. The individuals on networks interact with each other for several possible reasons, such as periodic contact or reply to former interactions. Regarding these interaction events as expectations based on previous interactions is crucial for understanding the underlying network and the corresponding dynamics. Usually, any change on individuals of the network will reflect on the pattern of their interaction events. However, the causes and expressed patterns for interaction events on networks have not been properly considered in network models. This article proposes a dynamic model for interaction event networks based on the temporal point process, which aims to incorporate the impact from historical interaction events on later interaction events considering both network structure and node connections. A network representation learning method is developed to learn the interaction event processes. The proposed interaction event network model also provides a convenient representation of the rate of interaction events for any pair of sender–receiver nodes on the network and therefore facilitates monitoring such event networks by summarizing these pairwise rates. Both simulation experiments and experiments on real-world data validate the effectiveness of the proposed model and the corresponding network representation learning algorithm.

Learning flexible network representation via anonymous walks

Network representation learning aims to project nodes in a network into low-dimensional continuous vector space while preserving the network structure and inherent node properties. Most of the existing approaches focus merely on the local context of nodes but ignore the local structural patterns of nodes that are important in network analysis tasks. A flexible network representation learning algorithm that can be generalized across a variety of domains and tasks should conform to two principles: to learn representations where nodes sharing similar local contexts have similar vectors and to embed nodes sharing similar local structures closely together. In this paper, we propose a flexible framework that incorporates local structural information into a generic model capturing contextual information. Specifically, anonymous walks are exploited to capture local structural patterns. In addition, we design two strategies to incorporate local structural patterns into the basic continuous bag-of-words (CBOW) architecture through statistic-based structural similarity and embedding-based structural vectors. Extensive experiments on real-world network data sets indicate that the proposed models incorporating local structural patterns outperform seven state-of-the-art network representation learning models in various tasks.

A survey on heterogeneous network representation learning

Heterogeneous information networks usually contain different kinds of nodes and distinguishing types of relations, which can preserve more information than homogeneous information networks. Heterogeneous network representation learning attempts to learn a low-dimensional representation for each node and capture rich semantic information of the given network. Most of existing surveys focus on heterogeneous information network analysis and homogeneous information network representation learning. Although considerable research efforts concentrate on heterogeneous network representation learning, there are few surveys that systematically review the state-of-the-art heterogeneous network representation learning techniques. Motivated by this, we propose a taxonomy of heterogeneous network representation learning algorithms according to different approaches of capturing semantic information in heterogeneous networks, including path based algorithms and semantic unit based algorithms. Moreover, we introduce the typical heterogeneous network representation learning techniques in detail and make a comparative analysis of these techniques. In addition, the research challenges in terms of semantics preserving, data sparsity and scalability are discussed. To tackle these challenges, several potential future research directions for heterogeneous network representation learning are pointed out, including semantic relations extraction, dynamic heterogeneous networks, very large heterogeneous networks and heterogeneous networks construction.