Heterogeneous Network Representation Learning Research Articles

Quintuple-based Representation Learning for Bipartite Heterogeneous Networks

Recent years have seen rapid progress in network representation learning, which removes the need for burdensome feature engineering and facilitates downstream network-based tasks. In reality, networks often exhibit heterogeneity, which means there may exist multiple types of nodes and interactions. Heterogeneous networks raise new challenges to representation learning, as the awareness of node and edge types is required. In this paper, we study a basic building block of general heterogeneous networks, the heterogeneous networks with two types of nodes. Many problems can be solved by decomposing general heterogeneous networks into multiple bipartite ones. Recently, to overcome the demerits of non-metric measures used in the embedding space, metric learning-based approaches have been leveraged to tackle heterogeneous network representation learning. These approaches first generate triplets of samples, in which an anchor node, a positive counterpart and a negative one co-exist, and then try to pull closer positive samples and push away negative ones. However, when dealing with heterogeneous networks, even the simplest two-typed ones, triplets cannot simultaneously involve both positive and negative samples from different parts of networks. To address this incompatibility of triplet-based metric learning, in this paper, we propose a novel quintuple-based method for learning node representations in bipartite heterogeneous networks. Specifically, we generate quintuples that contain positive and negative samples from two different parts of networks. And we formulate two learning objectives that accommodate quintuple-based learning samples, a proximity-based loss that models the relations in quintuples by sigmoid probabilities, and an angular loss that more robustly maintains similarity structures. In addition, we also parameterize feature learning by using one-dimensional convolution operators around nodes’ neighborhoods. Compared with eight methods, extensive experiments on two downstream tasks manifest the effectiveness of our approach.

Temporal Protein Complex Identification Based on Dynamic Heterogeneous Protein Information Network Representation Learning.

Protein complexes, as the fundamental units of cellular function and regulation, play a crucial role in understanding the normal physiological functions of cells. Existing methods for protein complex identification attempt to introduce other biological information on top of the protein-protein interaction (PPI) network to assist in evaluating the degree of association between proteins. However, these methods usually treat protein interaction networks as flat homogeneous static networks. They cannot distinguish the roles and importance of different types of biological information, nor can they reflect the dynamic changes of protein complexes. In recent years, heterogeneous network representation learning has achieved great success in processing complex heterogeneous information and mining deep semantics. We thus propose a temporal protein complex identification method based on Dynamic Heterogeneous Protein information network Representation Learning, DHPRL. DHPRL naturally integrates multiple types of heterogeneous biological information in the cellular temporal dimension. It simultaneously models the temporal dynamic properties of proteins and the heterogeneity of biological information to improve the understanding of protein interactions and the accuracy of complex prediction. Firstly, we construct Dynamic Heterogeneous Protein Information Network (DHPIN) by integrating temporal gene expression information and GO attribute information. Then we design a dual-view collaborative contrast mechanism. Specifically, proposing to learn protein representations from two views of DHPIN (1-hop relation view and meta-path view) to model the consistency and specificity between nearest-neighbour bio information and deeper biological semantics. The dynamic PPI network is thereafter re-weighted based on the learned protein representations. Finally, we perform protein identification on the re-weighted dynamic PPI network. Extensive experimental results demonstrate that DHPRL can effectively model complicated biological information and achieve state-of-the-art performance in most cases. The source code and datasets for DHPR are available at https://github.com/LI-jasm/DHPRL.

Multi-view learning-based heterogeneous network representation learning

Network representation learning is an important tool for extracting latent features from heterogeneous networks to enhance downstream analysis tasks. However, for heterogeneous networks in the era of big data, their heterogeneity, unseen network noises, latent structural and semantic features, and the coupling degree between the two features are increasing. These unignorable changes lead to the difficulties of extracting latent features and the demand for further optimization of existing embedding models. In this paper, an unsupervised multi-view learning based heterogeneous network representation model is proposed, called MVHNR. Firstly, to simplify the difficulty of feature learning, a novel view generation strategy is designed to divide the heterogeneous network into multiple sub-views that contain only one semantic feature and its related structural features. Then, to improve the accuracy and robustness of feature learning with network noises, a novel adjacency matrix strategy is constructed for each view, and a novel adversarial learning strategy is used to both learn the local semantic and structural features in each view. Finally, to aggregate the local features of all views, a multi-headed attention network is used to generate a global feature vector of each node. Extensive experimental results on three tasks and five public networks show the advantages of our MVHNR model.

Multiplex heterogeneous network representation learning with unipath based global awareness neural network

Network embedding has gained great popularity in tackling various network analytical tasks, such as link prediction and node classification. However, most existing works from heterogeneous networks ignore the relation heterogeneity with multiplex networks between multi-typed nodes. To tackle this challenge, this work proposes a Unipath based global Awareness neural Network (UAN) for attributed multiplex heterogeneous network embedding. Our UAN can automatically learn useful interactions of unipath networks and the base network. Additionally, we effectively integrate both multi-relation structural signals and attribute semantics into the learned node embeddings with both unsupervised and semi-supervised learning paradigms. Extensive experiments were conducted on real-world datasets in three different domains and various network analysis tasks were performed. Experimental results demonstrate the significant superiority of UAN against state-of-the-art embedding baselines in terms of all evaluation metrics.

Heterogeneous network representation learning based on role feature extraction

Since most of the real-world networks are heterogeneous, existing methods cannot characterize the roles of nodes in heterogeneous networks. The neighborhood structure of nodes in heterogeneous networks largely determines the node roles, and the basic statistical features of nodes describe the topology of nodes to some extent, so extracting structural features from the adjacency matrix of networks is crucial for role-oriented network representation learning(structural equivalence). Therefore, in this paper, we propose a heterogeneous network representation learning model based on role feature extraction, called HRFE(Heterogeneous Network Representation Learning for Role Feature Extraction). Firstly, we perform feature extraction for each node in the heterogeneous network to obtain a high-dimensional feature matrix, then perform role discovery using non-negative matrix decomposition techniques to obtain a role-based node representation, and finally verify the effectiveness of the model HRFE through experiments on a large number of real datasets.

Heterogeneous Network Representation Learning Approach for Ethereum Identity Identification

Recently, network representation learning has been widely used to mine and analyze network characteristics, and it is also applied to blockchain, but most of the embedding methods in blockchain ignore the heterogeneity of network, so it is difficult to accurately describe the characteristics of the transaction. As smart society evolves, Ethereum makes smart contracts reality, while the mine of transaction characteristics appearing on the Ethereum platform is scarce; thus, there is an urgent need to mine Ethereum from contract and transfer. In this article, we propose a heterogeneous network representation learning method to mine implicit information inside Ethereum transactions. Specifically, we construct an Ethereum transaction network by collecting transaction data from normal and phishing Ethereum accounts. Then, we propose a walk strategy that combines timestamps and transaction amounts to represent the information that occurs at the time of a transaction. To mine the types of nodes and edges, we use a heterogeneous network representation learning method to map the transaction network to a low-dimensional space. Finally, we improve the accuracy of the embedding results in the node classification task, which has important implications for Ethereum mining as well as identity recognition.

Prediction of potential small molecule-miRNA associations based on heterogeneous network representation learning.

MicroRNAs (miRNAs) are closely associated with the occurrences and developments of many complex human diseases. Increasing studies have shown that miRNAs emerge as new therapeutic targets of small molecule (SM) drugs. Since traditional experiment methods are expensive and time consuming, it is particularly crucial to find efficient computational approaches to predict potential small molecule-miRNA (SM-miRNA) associations. Considering that integrating multi-source heterogeneous information related with SM-miRNA association prediction would provide a comprehensive insight into the features of both SMs and miRNAs, we proposed a novel model of Small Molecule-MiRNA Association prediction based on Heterogeneous Network Representation Learning (SMMA-HNRL) for more precisely predicting the potential SM-miRNA associations. In SMMA-HNRL, a novel heterogeneous information network was constructed with SM nodes, miRNA nodes and disease nodes. To access and utilize of the topological information of the heterogeneous information network, feature vectors of SM and miRNA nodes were obtained by two different heterogeneous network representation learning algorithms (HeGAN and HIN2Vec) respectively and merged with connect operation. Finally, LightGBM was chosen as the classifier of SMMA-HNRL for predicting potential SM-miRNA associations. The 10-fold cross validations were conducted to evaluate the prediction performance of SMMA-HNRL, it achieved an area under of ROC curve of 0.9875, which was superior to other three state-of-the-art models. With two independent validation datasets, the test experiment results revealed the robustness of our model. Moreover, three case studies were performed. As a result, 35, 37, and 22 miRNAs among the top 50 predicting miRNAs associated with 5-FU, cisplatin, and imatinib were validated by experimental literature works respectively, which confirmed the effectiveness of SMMA-HNRL. The source code and experimental data of SMMA-HNRL are available at https://github.com/SMMA-HNRL/SMMA-HNRL.

Multiple heterogeneous network representation learning based on multi-granularity fusion

Multiple heterogeneous network representation learning based on multi-granularity fusion

Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark

Since real-world objects and their interactions are often multi-modal and multi-typed, heterogeneous networks have been widely used as a more powerful, realistic, and generic superclass of traditional homogeneous networks (graphs). Meanwhile, representation learning ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a.k.a.</i> embedding) has recently been intensively studied and shown effective for various network mining and analytical tasks. In this work, we aim to provide a unified framework to deeply summarize and evaluate existing research on heterogeneous network embedding (HNE), which includes but goes beyond a normal survey. Since there has already been a broad body of HNE algorithms, as the first contribution of this article, we provide a generic paradigm for the systematic categorization and analysis over the merits of various existing HNE algorithms. Moreover, existing HNE algorithms, though mostly claimed generic, are often evaluated on different datasets. Understandable due to the application favor of HNE, such indirect comparisons largely hinder the proper attribution of improved task performance towards effective data preprocessing and novel technical design, especially considering the various ways possible to construct a heterogeneous network from real-world application data. Therefore, as the second contribution, we create four benchmark datasets with various properties regarding scale, structure, attribute/label availability, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> . from different sources, towards handy and fair evaluations of HNE algorithms. As the third contribution, we carefully refactor and amend the implementations and create friendly interfaces for 13 popular HNE algorithms, and provide all-around comparisons among them over multiple tasks and experimental settings. By putting all existing HNE algorithms under a unified framework, we aim to provide a universal reference and guideline for the understanding and development of HNE algorithms. Meanwhile, by open-sourcing all data and code, we envision to serve the community with an ready-to-use benchmark platform to test and compare the performance of existing and future HNE algorithms ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yangji9181/HNE</uri> ).

CircWalk: a novel approach to predict CircRNA-disease association based on heterogeneous network representation learning

BackgroundSeveral types of RNA in the cell are usually involved in biological processes with multiple functions. Coding RNAs code for proteins while non-coding RNAs regulate gene expression. Some single-strand RNAs can create a circular shape via the back splicing process and convert into a new type called circular RNA (circRNA). circRNAs are among the essential non-coding RNAs in the cell that involve multiple disorders. One of the critical functions of circRNAs is to regulate the expression of other genes through sponging micro RNAs (miRNAs) in diseases. This mechanism, known as the competing endogenous RNA (ceRNA) hypothesis, and additional information obtained from biological datasets can be used by computational approaches to predict novel associations between disease and circRNAs.ResultsWe applied multiple classifiers to validate the extracted features from the heterogeneous network and selected the most appropriate one based on some evaluation criteria. Then, the XGBoost is utilized in our pipeline to generate a novel approach, called CircWalk, to predict CircRNA-Disease associations. Our results demonstrate that CircWalk has reasonable accuracy and AUC compared with other state-of-the-art algorithms. We also use CircWalk to predict novel circRNAs associated with lung, gastric, and colorectal cancers as a case study. The results show that our approach can accurately detect novel circRNAs related to these diseases.ConclusionsConsidering the ceRNA hypothesis, we integrate multiple resources to construct a heterogeneous network from circRNAs, mRNAs, miRNAs, and diseases. Next, the DeepWalk algorithm is applied to the network to extract feature vectors for circRNAs and diseases. The extracted features are used to learn a classifier and generate a model to predict novel CircRNA-Disease associations. Our approach uses the concept of the ceRNA hypothesis and the miRNA sponge effect of circRNAs to predict their associations with diseases. Our results show that this outlook could help identify CircRNA-Disease associations more accurately.

Heterogeneous information networks

In 2011, we proposed PathSim to systematically define and compute similarity between nodes in a heterogeneous information network (HIN), where nodes and links are from different types. In the PathSim paper, we for the first time introduced HIN with general network schema and proposed the concept of meta-paths to systematically define new relation types between nodes. In this paper, we summarize the impact of PathSim paper in both academia and industry. We start from the algorithms that are based on meta-path-based feature engineering, then move on to the recent development in heterogeneous network representation learning, including both shallow network embedding and heterogeneous graph neural networks. In the end, we make the connection between knowledge graphs and HINs and discuss the implication of meta-paths in the symbolic reasoning scenario. Finally, we point out several future directions.

MGRL: attributed multiplex heterogeneous network representation learning based on multi-granularity information fusion

MGRL: attributed multiplex heterogeneous network representation learning based on multi-granularity information fusion

Rating prediction model based on heterogeneous network representation learning

Rating prediction model based on heterogeneous network representation learning

A survey on heterogeneous network representation learning

Heterogeneous information networks usually contain different kinds of nodes and distinguishing types of relations, which can preserve more information than homogeneous information networks. Heterogeneous network representation learning attempts to learn a low-dimensional representation for each node and capture rich semantic information of the given network. Most of existing surveys focus on heterogeneous information network analysis and homogeneous information network representation learning. Although considerable research efforts concentrate on heterogeneous network representation learning, there are few surveys that systematically review the state-of-the-art heterogeneous network representation learning techniques. Motivated by this, we propose a taxonomy of heterogeneous network representation learning algorithms according to different approaches of capturing semantic information in heterogeneous networks, including path based algorithms and semantic unit based algorithms. Moreover, we introduce the typical heterogeneous network representation learning techniques in detail and make a comparative analysis of these techniques. In addition, the research challenges in terms of semantics preserving, data sparsity and scalability are discussed. To tackle these challenges, several potential future research directions for heterogeneous network representation learning are pointed out, including semantic relations extraction, dynamic heterogeneous networks, very large heterogeneous networks and heterogeneous networks construction.

Association Mining to Identify Microbe Drug Interactions Based on Heterogeneous Network Embedding Representation.

Accurately identifying microbe-drug associations plays a critical role in drug development and precision medicine. Considering that the conventional wet-lab method is time-consuming, labor-intensive and expensive, computational approach is an alternative choice. The increasing availability of numerous biological data provides a great opportunity to systematically understand complex interaction mechanisms between microbes and drugs. However, few computational methods have been developed for microbe drug prediction. In this work, we leverage multiple sources of biomedical data to construct a heterogeneous network for microbes and drugs, including drug-drug interactions, microbe-microbe interactions and microbe-drug associations. And then we propose a novel Heterogeneous Network Embedding Representation framework for Microbe-Drug Association prediction, named (HNERMDA), by combining metapath2vec with bipartite network recommendation. In this framework, we introduce metapath2vec, a heterogeneous network representation learning method, to learn low-dimensional embedding representations for microbes and drugs. Following that, we further design a bias bipartite network projection recommendation algorithm to improve prediction accuracy. Comprehensive experiments on two datasets, named MDAD and aBiofilm, demonstrated that our model consistently outperformed five baseline methods in three types of cross-validations. Case study on two popular drugs (i.e., Ciprofloxacin and Pefloxacin) further validated the effectiveness of our HNERMDA model in inferring potential target microbes for drugs.

A Semantic Aware Meta-Path Model for Heterogeneous Network Representation Learning

Heterogeneous graph representation learning is to learn effective representations for nodes or (sub)graphs, which preserve node attributes and structural information. However, it is challenging to design a representation learning method for heterogeneous information networks (HINs) due to their diversity. Most of the existing HIN-oriented learning methods define a series of meta-paths. Then, they aggregate the representations learned from different meta-paths in the same hidden space. These methods do not consider semantic differences of different meta-paths, which leads to semantic confusion. And further affects the effectiveness of the learned representation. Given these issues, we introduce a Semantic Aware HIN Representation learning Network (SAHRN), which takes into account the semantics of different meta-paths. We mitigate the problem of semantic confusion by projecting nodes’ features into different hidden spaces separately according to different meta-paths. To further expand the scope of aggregation and enrich the aggregated information, we also design various variants of our model by adding layer aggregation. Extensive experiments on three standard HIN datasets show that SAHRN achieves consistent improvements compared to state-of-the-art graph representation learning methods. The experiments and analyses on each component of the model show the effectiveness of the proposed method. The source code is available on https://github.com/pingpingand/SAHRN .

A name disambiguation module for intelligent robotic consultant in industrial internet of things

Abstract Nowadays, the industrial internet of things (IIoT) is becoming more and more popular and necessary in our daily life. Intelligent robotic consultant which is a very important application of IIoT has attracted the attention of researchers. Robotic literature consultant which aims at helping users find related news or articles are becoming more and more important. As far as we know that the name disambiguation problem of robotic literature consultant still remain unsolved. networks (e.g., Utilizing simple topological structure in academic network (e.g., the relationships of co-authors) and feature engineering are the main method. However, it is generally difficult to generate the features considering the information's privacy and availability. In addition, the semantics underlying the real-world academic data cannot be easily captured by the simple relationship data. Therefore, in this paper, a graph embedding based name disambiguation module named Mech-RL is introduced for the robotic literature consultant and a novel meta-path channel based heterogeneous network representation learning method named Mech-RL is proposed. Firstly, given some meaningful meta-paths, the heterogeneous information network and meta-path channels are extracted. Then, by proposing two meta-path based proximity measures, Mech-RL obtains the node embeddings by sampling and jointly updating according to each meta-path channel. Finally, we apply the clustering algorithm on the generated paper embeddings to complete the disambiguation task. The real-world dataset based experimental results demonstrate our approach is effective compared to the related author name disambiguation approaches.

W-MetaPath2Vec: The topic-driven meta-path-based model for large-scaled content-based heterogeneous information network representation learning

Recently, heterogeneous network representation learning has attracted a lot of attentions due to its potential applications. Our works in this paper are concentrated on how to leverage the output of network representation learning by combining with the topic similarity between nodes in content-based heterogeneous information network (CHIN). These unique challenges come from the shortage of topic similarity evaluation between text-based nodes which limit the accuracy of the similarity search as well other network mining tasks. Moreover, the massive sizes of current real-world network also raises challenges for traditional standalone-based heterogeneous network analysis models. Different from previous network representation learning models, such as: Node2Vec or Metapath2Vec, our proposed W-MethPath2Vec model uses the topic-driven meta-path-based random walk mechanism for generating heterogeneous neighborhood of nodes as the learning features. Then, these learning nodes’ features are used to train the learning model which is used for solving various heterogeneous network mining tasks such as: node similarity search, clustering, classification, link prediction, etc. The W-MethPath2Vec model enables the simultaneous modeling of structural and topic correlations between nodes in heterogeneous networks. Moreover, the W-MethPath2Vec model is implemented in the Apache Spark-based distributed framework which enables the capability of handling large-scaled networks. We tested our W-MethPath2Vec model with the previous state-of-the-art approaches in the real-world datasets to demonstrate the effectiveness of our proposed model.

DHNE: Network Representation Learning Method for Dynamic Heterogeneous Networks

Analyzing the rich information behind heterogeneous networks through network representation learning methods is signifcant for many application tasks such as link prediction, node classifcation and similarity research. As the networks evolve over times, the interactions among the nodes in networks make heterogeneous networks exhibit dynamic characteristics. However, almost all the existing heterogeneous network representation learning methods focus on static networks which ignore dynamic characteristics. In this paper, we propose a novel approach DHNE to learn the representations of nodes in dynamic heterogeneous networks. The key idea of our approach is to construct comprehensive historical-current networks based on subgraphs of snapshots in time step to capture both the historical and current information in the dynamic heterogeneous network. And then under the guidance of meta paths, DHNE performs random walks on the constructed historical-current graphs to capture semantic information. After getting the node sequences through random walks, we propose the dynamic heterogeneous skip-gram model to learn the embeddings. Experiments on large-scale real-world networks demonstrate that the embeddings learned by the proposed DHNE model achieve better performances than state-of-the-art methods in various downstream tasks including node classifcation and visualization.