Network Representation Learning Model Research Articles

A multi-source molecular network representation model for protein–protein interactions prediction

The prediction of potential protein–protein interactions (PPIs) is a critical step in decoding diseases and understanding cellular mechanisms. Traditional biological experiments have identified plenty of potential PPIs in recent years, but this problem is still far from being solved. Hence, there is urgent to develop computational models with good performance and high efficiency to predict potential PPIs. In this study, we propose a multi-source molecular network representation learning model (called MultiPPIs) to predict potential protein–protein interactions. Specifically, we first extract the protein sequence features according to the physicochemical properties of amino acids by utilizing the auto covariance method. Second, a multi-source association network is constructed by integrating the known associations among miRNAs, proteins, lncRNAs, drugs, and diseases. The graph representation learning method, DeepWalk, is adopted to extract the multisource association information of proteins with other biomolecules. In this way, the known protein–protein interaction pairs can be represented as a concatenation of the protein sequence and the multi-source association features of proteins. Finally, the Random Forest classifier and corresponding optimal parameters are used for training and prediction. In the results, MultiPPIs obtains an average 86.03% prediction accuracy with 82.69% sensitivity at the AUC of 93.03% under five-fold cross-validation. The experimental results indicate that MultiPPIs has a good prediction performance and provides valuable insights into the field of potential protein–protein interactions prediction. MultiPPIs is free available at https://github.com/jiboyalab/multiPPIs.

A Network Representation Learning Model Based on Multiple Remodeling of Node Attributes

Current network representation learning models mainly use matrix factorization-based and neural network-based approaches, and most models still focus only on local neighbor features of nodes. Knowledge representation learning aims to learn low-dimensional dense representations of entities and relations from structured knowledge graphs, and most models use the triplets to capture semantic, logical, and topological features between entities and relations. In order to extend the generalization capability of the network representation learning models, this paper proposes a network representation learning algorithm based on multiple remodeling of node attributes named MRNR. The model constructs the knowledge triplets through the textual association relationships between nodes. Meanwhile, a novel co-occurrence word training method has been proposed. Multiple remodeling of node attributes can significantly improve the effectiveness of network representation learning. At the same time, MRNR introduces the attention mechanism to achieve the weight information for key co-occurrence words and triplets, which further models the semantic and topological features between entities and relations, and it makes the network embedding more accurate and has better generalization ability.

Heterogeneous network representation learning based on role feature extraction

Since most of the real-world networks are heterogeneous, existing methods cannot characterize the roles of nodes in heterogeneous networks. The neighborhood structure of nodes in heterogeneous networks largely determines the node roles, and the basic statistical features of nodes describe the topology of nodes to some extent, so extracting structural features from the adjacency matrix of networks is crucial for role-oriented network representation learning(structural equivalence). Therefore, in this paper, we propose a heterogeneous network representation learning model based on role feature extraction, called HRFE(Heterogeneous Network Representation Learning for Role Feature Extraction). Firstly, we perform feature extraction for each node in the heterogeneous network to obtain a high-dimensional feature matrix, then perform role discovery using non-negative matrix decomposition techniques to obtain a role-based node representation, and finally verify the effectiveness of the model HRFE through experiments on a large number of real datasets.

MRLBot: Multi-Dimensional Representation Learning for Social Media Bot Detection

Social media bots pose potential threats to the online environment, and the continuously evolving anti-detection technologies require bot detection methods to be more reliable and general. Current detection methods encounter challenges, including limited generalization ability, susceptibility to evasion in traditional feature engineering, and insufficient exploration of user relationships. To tackle these challenges, this paper proposes MRLBot, a social media bot detection framework based on unsupervised representation learning. We design a behavior representation learning model that utilizes Transformer and a CNN encoder–decoder to simultaneously extract global and local features from behavioral information. Furthermore, a network representation learning model is proposed that introduces intra- and outer-community-oriented random walks to learn structural features and community connections from the relationship graph. Finally, the behavioral representation and relationship representation learning models are combined to generate fused representations for bot detection. The experimental results of four publicly available social network datasets demonstrate that the proposed method has certain advantages over state-of-the-art detection methods in this field.

Dual adaptive learning multi-task multi-view for graph network representation learning

Graph network analysis, which achieves widely application, is to explore and mine the graph structure data. However, existing graph network analysis methods with graph representation learning technique ignore the correlation between multiple graph network analysis tasks, and they need massive repeated calculation to obtain each graph network analysis results. Or they cannot adaptively balance the relative importance of multiple graph network analysis tasks, that lead to weak model fitting. Besides, most of existing methods ignore multiplex views semantic information and global graph information, which fail to learn robust node embeddings resulting in unsatisfied graph analysis results. To solve these issues, we propose a multi-task multi-view adaptive graph network representation learning model, called M2agl. The highlights of M2agl are as follows: (1) Graph convolutional network with the linear combination of the adjacency matrix and PPMI (positive point-wise mutual information) matrix is utilized as encoder to extract the local and global intra-view graph feature information of the multiplex graph network. Each intra-view graph information of the multiplex graph network can adaptively learn the parameters of graph encoder. (2) We use regularization to capture the interaction information among different graph views, and the importance of different graph views are learned by view attention mechanism for further inter-view graph network fusion. (3) The model is trained oriented by multiple graph network analysis tasks. The relative importance of multiple graph network analysis tasks are adjusted adaptively with the homoscedastic uncertainty. The regularization can be considered as an auxiliary task to further boost the performance. Experiments on real-worlds attributed multiplex graph networks demonstrate the effectiveness of M2agl in comparison with other competing approaches.

Improving Link Prediction in Network Representation Learning with Feature Fusion and Local Outlier Factor

Complex networks are a diverse set of networks found in various fields, such as social, technological, and biological networks. One important task in complex network analysis is link prediction, which involves detecting missing links or predicting future link formation. Many methods based on network structure analysis have been developed for link prediction, including network representation learning (NRL) models that represent nodes in a low-dimensional space. Fusion-based attributed NRL methods are particularly effective, as they capture both content and structure information. However, NRL models for link prediction are binary classification models, which face challenges in identifying negative links and prioritizing predicted links. To address these challenges, we propose a novel approach that treats link prediction as a novelty detection problem. Our approach uses the Local Outlier Factor (LOF) algorithm to quantify the novelty of non-existent links based on the representations of existing links. Our experimental results show that our proposed approach outperforms existing methods, particularly when used with fusion-based attributed NRL models

ComGCN: Community-Driven Graph Convolutional Network for Link Prediction in Dynamic Networks

Recent advances in deep learning have tremendously leveraged the performance of network representation learning (NRL). Multiple deep learning-based NRL models have been proposed recently to effectively handling primitive tasks of information network analysis and mining (INAM) domain, including link prediction (LP). LP is considered as an important one due to its multiple applications in many disciplines. In the recent few years, LP in dynamic networks has attracted a lot of attention from researchers to propose novel algorithms for better capturing both rich structural and evolutional information of complex information networks (INs). However, recent models are mainly concentrated on preserving the sequential representations of a given network over time. They have largely ignored other important structural features, such as: intracommunity which contributes to the creation of links between network nodes. In this article, we propose a novel community-driven dynamic NRL technique upon the RNN+GCN framework, called: ComGCN. Specifically, the ComGCN model is a combination of microscopic (node embedding-based) and mesoscopic (intracommunity-based) dynamic network embedding approach which enable effectively handling the LP problem in context of dynamism. Extensive experiments on real-world dynamic networks demonstrated the effectiveness of the proposed model compared with recent state-of-the-art baselines.

CRL: Collaborative Representation Learning by Coordinating Topic Modeling and Network Embeddings.

Network representation learning (NRL) has shown its effectiveness in many tasks, such as vertex classification, link prediction, and community detection. In many applications, vertices of social networks contain textual information, e.g., citation networks, which form a text corpus and can be applied to the typical representation learning methods. The global context in the text corpus can be utilized by topic models to discover the topic structures of vertices. Nevertheless, most existing NRL approaches focus on learning representations from the local neighbors of vertices and ignore the global structure of the associated textual information in networks. In this article, we propose a unified model based on matrix factorization (MF), named collaborative representation learning (CRL), which: 1) considers complementary global and local information simultaneously and 2) models topics and learns network embeddings collaboratively. Moreover, we incorporate the Fletcher-Reeves (FR) MF, a conjugate gradient method, to optimize the embedding matrices in an alternative mode. We call this parameter learning method as AFR in our work that can achieve convergence after a few numbers of iterations. Also, by evaluating CRL on topic coherence and vertex classification using several real-world data sets, our experimental study shows that this collaborative model not only can improve the performance of topic discovery over the baseline topic models but also can learn better network representations than the state-of-the-art context-aware NRL models.

Leveraging network representation learning and community detection for analyzing the activity profiles of adolescents

Human mobility analysis plays a crucial role in urban analysis, city planning, epidemic modeling, and even understanding neighborhood effects on individuals’ health. Often, these studies model human mobility in the form of co-location networks. We have recently seen the tremendous success of network representation learning models on several machine learning tasks on graphs. To the best of our knowledge, limited attention has been paid to identifying communities using network representation learning methods specifically for co-location networks. We attempt to address this problem and study user mobility behavior through the communities identified with latent node representations. Specifically, we select several diverse network representation learning models to identify communities from a real-world co-location network. We include both general-purpose representation models that make no assumptions on network modality as well as approaches designed specifically for human mobility analysis. We evaluate these different methods on data collected in the Adolescent Health and Development in Context study. Our experimental analysis reveals that a recently proposed method (LocationTrails) offers a competitive advantage over other methods with respect to its ability to represent and reflect community assignment that is consistent with extant findings regarding neighborhood racial and socio-economic differences in mobility patterns. We also compare the learned activity profiles of individuals by factoring in their residential neighborhoods. Our analysis reveals a significant contrast in the activity profiles of individuals residing in white-dominated versus black-dominated neighborhoods and advantaged versus disadvantaged neighborhoods in a major metropolitan city of United States. We provide a clear rationale for this contrastive pattern through insights from the sociological literature.

PF-HIN:Pre-Training for Heterogeneous Information Networks

In network representation learning we learn how to represent heterogeneous information networks in a low-dimensional space so as to facilitate effective search, classification, and prediction solutions. Previous network representation learning methods typically require sufficient task-specific labeled data to address domain-specific problems. The trained model usually cannot be transferred to out-of-domain datasets. We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network. Unlike traditional network representation learning models that have to train the entire model all over again for every downstream task and dataset, PF-HIN only needs to fine-tune the model and a small number of extra task-specific parameters, thus improving model efficiency and effectiveness. During pre-training, we first transform the neighborhood of a given node into a sequence. PF-HIN is pre-trained based on two self-supervised tasks, masked node modeling and adjacent node prediction. We adopt deep bi-directional transformer encoders to train the model, and leverage factorized embedding parameterization and cross-layer parameter sharing to reduce the parameters. In the fine-tuning stage, we choose four benchmark downstream tasks, i.e., link prediction, similarity search, node classification, and node clustering. PF-HIN outperforms state-of-the-art alternatives on each of these tasks, on four datasets.

ScratchGAN: Network representation learning for scratch with preference‐based generative adversarial nets

With the rapid increase of users, Scratch, as a popular online social and programming platform, has accumulated massive project resources and complex relations across its social and programming learning network. However, it is challenging to utilize the network information for providing Scratch users with personalized services, despite there are already some useful network representation learning models. In this paper, a network representation learning model with preference-based generative adversarial nets for Scratch (ScratchGAN) is proposed to resolve this problem. In ScratchGAN, we first design a node-vector initialization approach to preserve structure information and side information of Scratch network. Then, considering to learn the fine-grained user preference information of network, we propose a novel Scratch adversarial learning model which includes a Scratch generative adversarial net and a user preference difference constraint component. The former aims to capture user preferences through a new generating strategy based on the delivery nature of preference. The latter attempts to embed users' detailed preference differences according to their interaction behaviors. ScratchGAN can mine user preferences while preserving network structure information and side information. Extensive experiments on the Scratch network show that ScratchGAN outperforms other state-of-the-art models in link prediction and recommendation tasks.

Discrete embedding for attributed graphs

Attributed graphs refer to graphs where both node links and node attributes are observable for analysis. Attributed graph embedding enables joint representation learning of node links and node attributes. Different from classical graph embedding methods such as Deepwalk and node2vec that first project node links into low-dimensional vectors which are then linearly concatenated with node attribute vectors as node representation, attributed graph embedding fully explores data dependence between node links and attributes by either using node attributes as class labels to supervise structure learning from node links, or reversely using node links to supervise the learning from node attributes. However, existing attributed graph embedding models are designed in continuous Euclidean spaces which often introduce data redundancy and impose challenges to storage and computation costs. In this paper, we study a new problem of discrete embedding for attributed graphs that can learn succinct node representations. Specifically, we present a Binarized Attributed Network Embedding model (BANE for short) to learn binary node representation by factorizing a Weisfeiler-Lehman proximity matrix under the constraint of binary node representation. Furthermore, based on BANE, we propose a new Low-bit Quantization for Attributed Network Representation learning model (LQANR for short) to learn even more compact node representation of bit-width values. Theoretical analysis and empirical studies on real-world datasets show that the new discrete embedding models outperform benchmark methods.

Codee: A Tensor Embedding Scheme for Binary Code Search

Given a target binary function, the binary code search retrieves top-K similar functions in the repository, and similar functions represent that they are compiled from the same source codes. Searching binary code is particularly challenging due to large variations of compiler tool-chains and options and CPU architectures, as well as thousands of binary codes. Furthermore, there are some pivotal issues in current binary code search schemes, including inaccurate text-based or token-based analysis, slow graph matching, or complex deep learning processes. In this paper, we present an unsupervised tensor embedding scheme, Codee, to carry out code search efficiently and accurately at the binary function level. First, we use an NLP-based neural network to generate the semantic-aware token embedding. Second, we propose an efficient basic block embedding generation algorithm based on the network representation learning model. We learn both the semantic information of instructions and the control flow structural information to generate the basic block embedding. Then we use all basic block embeddings in a function to obtain a variable-length function feature vector. Third, we build a tensor to generate function embeddings based on the tensor singular value decomposition, which compresses the variable-length vectors into short fixed-length vectors to facilitate efficient search afterward. We further propose a dynamic tensor compression algorithm to incrementally update the function embedding database. Finally, we use the local sensitive hash method to find the top- <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> similar matching functions in the repository. Compared with state-of-the-art cross-optimization-level code search schemes, such as Asm2Vec and DeepBinDiff, our scheme achieves higher average search accuracy, shorter feature vectors, and faster feature generation performance using four datasets, OpenSSL, Coreutils, libgmp and libcurl. Compared with other cross-platform and cross-optimization-level code search schemes, such as Gemini, Safe, the average recall of our method also outperforms others.

Learning flexible network representation via anonymous walks

Network representation learning aims to project nodes in a network into low-dimensional continuous vector space while preserving the network structure and inherent node properties. Most of the existing approaches focus merely on the local context of nodes but ignore the local structural patterns of nodes that are important in network analysis tasks. A flexible network representation learning algorithm that can be generalized across a variety of domains and tasks should conform to two principles: to learn representations where nodes sharing similar local contexts have similar vectors and to embed nodes sharing similar local structures closely together. In this paper, we propose a flexible framework that incorporates local structural information into a generic model capturing contextual information. Specifically, anonymous walks are exploited to capture local structural patterns. In addition, we design two strategies to incorporate local structural patterns into the basic continuous bag-of-words (CBOW) architecture through statistic-based structural similarity and embedding-based structural vectors. Extensive experiments on real-world network data sets indicate that the proposed models incorporating local structural patterns outperform seven state-of-the-art network representation learning models in various tasks.

Topic-aware latent models for representation learning on networks

Network representation learning (NRL) methods have received significant attention over the last years thanks to their success in several graph analysis problems, including node classification, link prediction, and clustering. Such methods aim to map each vertex of the network into a low-dimensional space in a way that the structural information of the network is preserved. Of particular interest are methods based on random walks; such methods transform the network into a collection of node sequences, aiming to learn node representations by predicting the context of each node within the sequence. In this paper, we introduce TNE, a generic framework to enhance the embeddings of nodes acquired by means of random walk-based approaches with topic-based information. Similar to the concept of topical word embeddings in Natural Language Processing, the proposed model first assigns each node to a latent community with the favor of various statistical graph models and community detection methods and then learns the enhanced topic-aware representations. We evaluate our methodology in two downstream tasks: node classification and link prediction. The experimental results demonstrate that by incorporating node and community embeddings, we are able to outperform widely-known baseline NRL models.

Deep attributed network representation learning of complex coupling and interaction

Networks that can describe complex systems in nature are increasingly coupled and interacted, and effective modeling on complex coupling and interaction information is an important research direction of artificial intelligence. Representation learning provides us with a paradigm to solve such issues, but the current network representation learning methods are difficult to capture the coupling and interaction information in complex networks. In this paper, we propose a novel deep attributed network representation learning model framework (RolEANE), which can effectively preserve the highly nonlinear coupling and interactive network topological structure and attribute information. We design two different structural role proximity enhancement strategies for the deep autoencoder in the model framework, so that it can efficiently capture network topological structure and attribute information. In addition, the neighbor-modified Skip-Gram model in our model framework can efficiently and seamlessly integrate network topological structure and attribute information, and the selection of an appropriate representation learning output strategy can significantly improve the final performance of the algorithm. The experiments on four real datasets show that our method consistently outperforms the state-of-the-art network representation learning methods. On the node classification task, the average performance is improved by 4.52%–10.28% than the optimal baseline method; on the link prediction task, the average performance is 4.63% higher than the optimal baseline method.

HSNR: A Network Representation Learning Algorithm Using Hierarchical Structure Embedding

Vertices in the same group tend to connect densely, and usually share common attributes. Groups of different sizes reflect the relations of vertices in different ranges, and also reflect the features of different orders of the network. In this work, we propose a novel network representation learning algorithm by introducing group features of vertices of different orders to learn more discriminative network representations, named as network representation learning algorithm using Hierarchical structure embedding (HSNR). HSNR algorithm firstly constructs hierarchical relations of network structures of different orders based on greedy algorithm and modularity. In order to introduce hierarchical features into the network representation learning model, HSNR algorithm then introduces the idea of multi-relational modeling from knowledge representation, and converts the hierarchical relations into the triplet form between vertices. Finally, HSNR proposes a joint learning model embedding vertex triplets into the network representations. The experimental results show that the HSRN algorithm presented has an excellent performance in network vertex classification task on three real-world datasets.

Learning Embeddings Based on Global Structural Similarity in Heterogeneous Networks

With different types of nodes and edges, heterogeneous networks have higher levels of structural diversity than homogeneous networks. This article proposes an unsupervised representation learning model, named gs2vec, to address structural diversity of a node being connected to other types of nodes via different types of edges in heterogeneous networks. The model measures a node's structural roles based on its numbers of neighboring nodes of different types. It also attempts to measure such structural roles beyond the immediate neighborhood of each node by incorporating structural roles of other nodes k-hop away. Experiments based on synthetic and empirical datasets show that gs2vec outperforms state-of-the-art network representation learning models in heterogeneous network analysis tasks such as node classification and node clustering.

Rich heterogeneous information preserving network representation learning

Network representation learning has attracted increasing attention recently due to its applicability in network analysis. However, most existing network representation learning models only focus on preserving fragmentary aspects of network information, either node proximities or fixed semantic information. In this paper, we propose a novel algorithm named Rich Heterogeneous Information Preserving Network Representation Learning (HIRL), which integrates the high-order proximity among nodes and semantic information into a generic framework by exploiting a flexible autoencoder network. Based on the proposed algorithm, we can explore the hidden information in heterogeneous information networks through any custom form of path schema, and represents different types of nodes in a continuous and common vector space. Moreover, the proposed HIRL is applicable to homogeneous information networks. Extensive experimental results demonstrate that our approach can effectively preserve the information in networks under various path schemas, and performs better on real-world applications such as network reconstruction, link prediction, and node classification compared with the state-of-the-art methods.

Learning Effective Feature Representation against User Privacy Protection on Social Networks

Users pay increasing attention to their data privacy in online social networks, resulting in hiding personal information, such as profile attributes and social connections. While network representation learning (NRL) is widely effective in social network analysis (SNA) tasks, it is essential to learn effective node embeddings from privacy-protected sparse and incomplete network data. In this work, we present a novel NRL model to generate node embeddings that can afford data incompleteness coming from user privacy protection. We propose a structure-attribute enhanced matrix (SAEM) to alleviate data sparsity and develop a community-cluster informed NRL method, c2n2v, to further improve the quality of embedding learning. Experiments conducted across three datasets, three simulations of user privacy protection, and three downstream SNA tasks exhibit the promising performance of our NRL model SAEM+c2n2v.