Network Mining Tasks Research Articles

Temporal Network Embedding Enhanced With Long-Range Dynamics and Self-Supervised Learning.

Temporal network embedding (TNE) has promoted the research of knowledge discovery and reasoning on networks. It aims to embed vertices of temporal networks into a low-dimensional vector space while preserving network structures and temporal properties. However, most existing methods have limitations in capturing dynamics over long distances, which makes it difficult to explore multihop topological associations among vertices. To tackle this challenge, we propose LongTNE, which learns the long-range dynamics of vertices to endow TNE with the ability to capture high-order proximity (HP) of networks. In LongTNE, we employ graph self-supervised learning (Graph SSL) to optimize the establishment probability of deep links in each network snapshot. We also present an accumulated forward update (AFU) module to fathom global temporal evolution among multiple network snapshots. The empirical results on six temporal networks demonstrate that, in addition to achieving state-of-the-art performance on network mining tasks, LongTNE can be handily extended to existing TNE methods.

Attributed network embedding based on self-attention mechanism for recommendation method

Network embedding is a technique used to learn a low-dimensional vector representation for each node in a network. This method has been proven effective in network mining tasks, especially in the area of recommendation systems. The real-world scenarios often contain rich attribute information that can be leveraged to enhance the performance of representation learning methods. Therefore, this article proposes an attribute network embedding recommendation method based on self-attention mechanism (AESR) that caters to the recommendation needs of users with little or no explicit feedback data. The proposed AESR method first models the attribute combination representation of items and then uses a self-attention mechanism to compactly embed the combination representation. By representing users as different anchor vectors, the method can efficiently learn their preferences and reconstruct them with few learning samples. This achieves accurate and fast recommendations and avoids data sparsity problems. Experimental results show that AESR can provide personalized recommendations even for users with little explicit feedback information. Moreover, the attribute extraction of documents can effectively improve recommendation accuracy on different datasets. Overall, the proposed AESR method provides a promising approach to recommendation systems that can leverage attribute information for better performance.

MuxGNN: Multiplex Graph Neural Network for Heterogeneous Graphs.

Graph neural networks (GNNs) have become effective learning techniques for many downstream network mining tasks including node and graph classification, link prediction, and network reconstruction. However, most GNN methods have been developed for homogeneous networks with only a single type of node and edge. In this work we present muxGNN, a multiplex graph neural network for heterogeneous graphs. To model heterogeneity, we represent graphs as multiplex networks consisting of a set of relation layer graphs and a coupling graph that links node instantiations across multiple relations. We parameterize relation-specific representations of nodes and design a novel coupling attention mechanism that models the importance of multi-relational contexts for different types of nodes and edges in heterogeneous graphs. We further develop two complementary coupling structures: node invariant coupling suitable for node- and graph-level tasks, and node equivariant coupling suitable for link-level tasks. Extensive experiments conducted on six real-world datasets for link prediction in both transductive and inductive contexts and graph classification demonstrate the superior performance of muxGNN over state-of-the-art heterogeneous GNNs. In addition, we show that muxGNN's coupling attention discovers interpretable connections between different relations in heterogeneous networks.

A MOOC Course Data Analysis Based on an Improved Metapath2vec Algorithm

Many real-world scenarios can be naturally modeled as heterogeneous graphs, which contain both symmetry and asymmetry information. How to learn useful knowledge from the graph has become one of the hot spots of research in artificial intelligence. Based on Metapath2vec algorithm, an improved Metapath2vec algorithm is presented, which combines Metapath random walk, used to capture semantics and structure information between different nodes of a heterogeneous network, and GloVe model to consider the advantage of global text representation. In order to verify the feasibility and effectiveness of the model, node clustering and link prediction experiments were conducted on the self-generated ideal dataset and the MOOC course data. The analysis of experimental data on these tasks shows that the Metapath–GloVe algorithm learns consistently better embedding of heterogeneous nodes, and the algorithm improves the node embedding performance to better characterize the heterogeneous network structure and learn the characteristics of nodes, which proves the effectiveness and scalability of the proposed method in heterogeneous network mining tasks. It is also shown through extensive experiments that the Metapath–GloVe algorithm is more efficient than the non-negative matrix decomposition algorithm (NMF), and it can obtain better clustering results and more accurate prediction results in the video recommendation task.

An Unsupervised Rapid Network Alignment Framework via Network Coarsening

Network alignment aims to identify the correspondence of nodes between two or more networks. It is the cornerstone of many network mining tasks, such as cross-platform recommendation and cross-network data aggregation. Recently, with the development of network representation learning techniques, researchers have proposed many embedding-based network alignment methods. The effect is better than traditional methods. However, several issues and challenges remain for network alignment tasks, such as lack of labeled data, mapping across network embedding spaces, and computational efficiency. Based on the graph neural network (GNN), we propose the URNA (unsupervised rapid network alignment) framework to achieve an effective balance between accuracy and efficiency. There are two phases: model training and network alignment. We exploit coarse networks to accelerate the training of GNN after first compressing the original networks into small networks. We also use parameter sharing to guarantee the consistency of embedding spaces and an unsupervised loss function to update the parameters. In the network alignment phase, we first use a once-pass forward propagation to learn node embeddings of original networks, and then we use multi-order embeddings from the outputs of all convolutional layers to calculate the similarity of nodes between the two networks via vector inner product for alignment. Experimental results on real-world datasets show that the proposed method can significantly reduce running time and memory requirements while guaranteeing alignment performance.

A Graph-Based Topic Modeling Approach to Detection of Irrelevant Citations

In the recent years, the academic paper influence analysis has been widely studied due to its potential applications in the multiple areas of science information metric and retrieval. By identifying the academic influence of papers, authors, etc., we can directly support researchers to easily reach academic papers. These recommended candidate papers are not only highly relevant with their desired research topics but also highly-attended by the research community within these topics. For very recent years, the rapid developments of academic networks, like Google Scholar, Research Gate, CiteSeerX, etc., have significantly boosted the number of new published papers annually. It also helps to strengthen the borderless cooperation between researchers who are interested on the same research topics. However, these current academic networks still lack the capabilities of provisioning researchers deeper into most-influenced papers. They also largely ignore quite/irrelevant papers, which are not fully related with their current interest topics. Moreover, the distributions of topics within these academic papers are considered as varying and it is difficult to extract the main concentrated topics in these papers. Thus, it leads to challenges for researchers to find their appropriated/high-qualified reference resources while doing researches. To overcome this limitation, in this paper, we proposed a novel approach of paper influence analysis through their content-based and citation relationship-based analyses within the biographical network. In order to effectively extract the topic-based relevance from papers, we apply the integrated graph-based citation relationship analysis with topic modeling approach to automatically learn the distributions of keyword-based labeled topics in forms of unsupervised learning approach, named as TopCite. Then, we base on the constructed graph-based paper–topic structure to identify their relevancy levels. Upon the identified relevancy levels between papers, we can support for improving the accuracy performance of other bibliographic network mining tasks, such as paper similarity measurement, recommendation, etc. Extensive experiments in real-world AMiner bibliographic dataset demonstrate the effectiveness of our proposed ideas in this paper.

Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark

Since real-world objects and their interactions are often multi-modal and multi-typed, heterogeneous networks have been widely used as a more powerful, realistic, and generic superclass of traditional homogeneous networks (graphs). Meanwhile, representation learning ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a.k.a.</i> embedding) has recently been intensively studied and shown effective for various network mining and analytical tasks. In this work, we aim to provide a unified framework to deeply summarize and evaluate existing research on heterogeneous network embedding (HNE), which includes but goes beyond a normal survey. Since there has already been a broad body of HNE algorithms, as the first contribution of this article, we provide a generic paradigm for the systematic categorization and analysis over the merits of various existing HNE algorithms. Moreover, existing HNE algorithms, though mostly claimed generic, are often evaluated on different datasets. Understandable due to the application favor of HNE, such indirect comparisons largely hinder the proper attribution of improved task performance towards effective data preprocessing and novel technical design, especially considering the various ways possible to construct a heterogeneous network from real-world application data. Therefore, as the second contribution, we create four benchmark datasets with various properties regarding scale, structure, attribute/label availability, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> . from different sources, towards handy and fair evaluations of HNE algorithms. As the third contribution, we carefully refactor and amend the implementations and create friendly interfaces for 13 popular HNE algorithms, and provide all-around comparisons among them over multiple tasks and experimental settings. By putting all existing HNE algorithms under a unified framework, we aim to provide a universal reference and guideline for the understanding and development of HNE algorithms. Meanwhile, by open-sourcing all data and code, we envision to serve the community with an ready-to-use benchmark platform to test and compare the performance of existing and future HNE algorithms ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yangji9181/HNE</uri> ).

Representation learning for temporal networks using temporal random walk and deep autoencoder

Network representation learning is a promising direction towards applying machine learning over graph-structured data. Most of the recent researches focus on embedding static networks to a low dimensional vector space and performing traditional network mining tasks using this latent space. But, to study many complex interactions between real-world entities, we need to model the data into a time-varying network (temporal network), where the edge connectivity patterns may vary with time. We focus on the problem of temporal network representation learning, where the network is represented with edges time-stamped with the time of interaction. We design a random surfing model for the temporal network using a non-homogeneous Markov chain to generate a node similarity matrix. Further, we perform non-linear dimensionality reduction on the node similarity matrix using a denoising deep autoencoder to generate node representations. We also evaluate the quality of the embeddings generated using a temporal link prediction benchmark.

Motif-Aware Adversarial Graph Representation Learning

Graph representation learning has been extensively studied in recent years. It has been proven effective in network analysis and mining tasks such as node classification and link prediction. Learning method based on neural network has become mainstream solution because it can efficiently preserve nonlinear characteristics of graph. However, more efficient ways to encode complex graph structures are still being explored. In this paper, for undirected graphs, we present a novel Motif-Aware Generative Adversarial Network (MotifGAN) model to learn graph representation based on a re-weighted graph, which unifies both the Motif higher order structure and the original lower order structure. Firstly, we obtain the motif adjacency matrix by motif mining, which encodes the Motif higher order structure. Then we couple the motif adjacency matrix with the adjacency matrix of the original graph to get a reweighted matrix named motif connectivity matrix. The motif connectivity distribution implicit in the motif connectivity matrix is the target structure information we need to preserve. How to preserve a wealth of structural information is a challenge. Inspired by related applications of GAN, we formulate a GAN model to generate embedding representations that satisfy the target structure conditions. The GAN model usually consists of two components: generator and discriminator. Our generator tries to approximate the motif connectivity distribution, while the discriminator detects whether the connectivity of two vertexes is from ground truth or generated by the generator. In adversarial training, both parts can alternately and iteratively boost their performance. Experimental results on public datasets show that MotifGAN has made substantial progress in various applications, including link prediction, node classification and visualization.

Multi-view Dynamic Heterogeneous Information Network Embedding

Abstract Most existing heterogeneous information network (HIN) embedding methods focus on static environments while neglecting the evolving characteristic of real-world networks. Although several dynamic embedding methods have been proposed, they are merely designed for homogeneous networks and cannot be directly applied in heterogeneous environments. To tackle above challenges, we propose a novel framework for incorporating temporal information into HIN embedding, named multi-view dynamic HIN embedding (MDHNE), which can efficiently preserve evolution patterns of implicit relationships from different views in updating node vectors over time. We first transform HIN to a series of homogeneous networks corresponding to different views. Then our proposed MDHNE applies recurrent neural network (RNN) to incorporate evolving pattern of complex network structure and semantic relationships between nodes into latent embedding spaces, and thus the node vectors from multiple views can be learned and updated when HIN evolves over time. Moreover, we come up with an attention-based fusion mechanism, which can automatically infer weights of latent vectors corresponding to different views by minimizing the objective function specific for different mining tasks. Extensive experiments clearly demonstrate that our MDHNE model outperforms state-of-the-art baselines on three real-world dynamic datasets for different network mining tasks.

CMG2Vec: A composite meta-graph based heterogeneous information network embedding approach

Heterogeneous information network embedding has been intensively studied in the past years. However, existing methods require users to manually assign meta-paths or meta-graphs in advance. Meanwhile, most of previous approaches only consider a single type of meta-path or meta-graph which is usually sparse and biased, and thus the node representations learned may be incomprehensive and inaccurate. To tackle these limitations, we proposed an extensible semantic description structure, called Composite Meta-Graph(CMG). By virtue of such a structure, users do not need to worry about selection of an appropriate meta-path or meta-graph. Rich semantic relations and rich structural contexts between nodes of different types and of different distances can be elaborated accurately according to CMG. Moreover, a CMG based heterogeneous information embedding framework, namely CMG2Vec, is also proposed. By expanding the auto-encoder into a heterogeneous network scenario, CMG2Vec can embed proximities between nodes of multiple orders learned from CMG into latent representations after a series of encoding–decoding non-linear mapping. During the fusing process, an attention mechanism is adopted to automatically learn weights of these latent vectors, which enables each final node representation to focus on proximity of the most informative order. Experimental results on three large-scale datasets demonstrate that our method outperforms existing state-of-the-art homogeneous and heterogeneous network embedding approaches in three network mining tasks in terms of node classification, node clustering, and node similarity search.

W-Com2Vec: A topic-driven meta-path- based intra-community embedding for content-based heterogeneous information network

Heterogeneous information network (HIN) are becoming popular across multiple applications in forms of complex large-scaled networked data such as social networks, bibliographic networks, biological networks, etc. Recently, information network embedding (INE) has aroused tremendously interests from researchers due to its effectiveness in information network analysis and mining tasks. From recent views of INE, community is considered as the mesoscopic preserving network’s structure which can be combined with traditional approach of network’s node proximities (microscopic structure preserving) to leverage the quality of network’s representation. Most of contemporary INE models, like as: HIN2Vec, Metapath2Vec, HINE, etc. mainly concentrate on microscopic network structure preserving and ignore the mesoscopic (intra-community) structure of HIN. In this paper, we introduce a novel approach of topic-driven meta-path-based embedding, namely W-Com2Vec (Weighted intra-community to vector). Our proposed W-Com2Vec model enables to capture richer semantic of node representation by applying the meta-path-based community-aware, node proximity preserving and topic similarity evaluation at the same time during the process of network embedding. We demonstrate comprehensive empirical studies on our proposed W-Com2Vec model with several real-world HINs. Experimental results show W-Com2Vec outperforms recent state-of-the-art INE models in solving primitive network analysis and mining tasks.

Adversarial network embedding using structural similarity

Network embedding which aims to embed a given network into a low-dimensional vector space has been proved effective in various network analysis and mining tasks such as node classification, link prediction and network visualization. The emerging network embedding methods have shifted of emphasis in utilizing mature deep learning models. The neural-network based network embedding has become a mainstream solution because of its high efficiency and capability of preserving the nonlinear characteristics of the network. In this paper, we propose Adversarial Network Embedding using Structural Similarity (ANESS), a novel, versatile, low-complexity GAN-based network embedding model which utilizes the inherent vertex-to-vertex structural similarity attribute of the network. ANESS learns robustness and effective vertex embeddings via a adversarial training procedure. Specifically, our method aims to exploit the strengths of generative adversarial networks in generating high-quality samples and utilize the structural similarity identity of vertexes to learn the latent representations of a network. Meanwhile, ANESS can dynamically update the strategy of generating samples during each training iteration. The extensive experiments have been conducted on the several benchmark network datasets, and empirical results demonstrate that ANESS significantly outperforms other state-of-the-art network embedding methods.

W-Metagraph2Vec: a novel approval of enriched schematic topic-driven heterogeneous information network embedding

Recently, heterogeneous information network (HIN) embedding is wide studied due to its various applications. In general, network embedding is a way of representation network’s nodes into a low-dimensional space. Most of previous embedding techniques concentrate on the homogeneous networks only in which all nodes are considered as a single type. Heterogeneous network embedding is a challenging problem due to the complexity of different node’s types and link’s types. Recent heterogeneous network embedding studies are based on meta-path and meta-graph to guide the random walks over the networks. These heterogeneous network embedding approaches outperform state-of-the-art homogeneous embedding models in multiple heterogeneous network mining tasks. However, recent meta-graph-based approaches are ineffective in capturing topic similarity between nodes. There is no doubt that most of common HINs (DBLP, Facebook, etc.) are rich-text which contain many text-based nodes, such as paper, comment, post, etc. In this paper, we propose a novel embedding approach, namely W-MetaGraph2Vec. The W-MetaGraph2Vec uses the topic-driven meta-graph-based random walk mechanism in weighted HIN to guide the generation of heterogeneous neighborhood of a node. Extensive experiments on real-world datasets demonstrate that our proposed model not only leverage HIN mining tasks, such as node similarity search, clustering, classification, etc. in performance accuracy but also discern the problems of topic relevance between text-based nodes.

Temporal Network Pattern Identification by Community Modelling

Temporal network mining tasks are usually hard problems. This is because we need to face not only a large amount of data but also its non-stationary nature. In this paper, we propose a method for temporal network pattern representation and pattern change detection following the reductionist approach. The main idea is to model each stable (durable) state of a given temporal network as a community in a sampled static network and the temporal state change is represented by the transition from one community to another. For this purpose, a reduced static single-layer network, called a target network, is constructed by sampling and rearranging the original temporal network. Our approach provides a general way not only for temporal networks but also for data stream mining in topological space. Simulation results on artificial and real temporal networks show that the proposed method can group different temporal states into different communities with a very reduced amount of sampled nodes.

Deep heterogeneous network embedding based on Siamese Neural Networks

Heterogeneous network embedding aims at mapping a heterogeneous network into a low-dimensional latent space. There exist diverse relations among different types of objects in heterogeneous networks. However, most existing heterogeneous network embedding methods focus on exploring network structures instead of relations among different objects, so some redundant and fuzzy relations are inevitably captured. To address the problem, we propose a Relation-Oriented Deep Embedding (RODE) framework for heterogeneous networks that explores different relations among nodes. The captured relations are modeled through node similarity and dissimilarity. Based on the similarity and dissimilarity, a multi-task Siamese Neural Network is formulated to perform network embedding and optimize embedding representations. Extensive experiments are conducted on four heterogeneous networks. Experimental results demonstrate our method outperforms state-of-the-art embedding algorithms on several network mining tasks, such as link prediction, node classification and node clustering.

Learning embeddings of a heterogeneous behavior network for potential behavior prediction

Potential behavior prediction involves understanding the latent human behavior of specific groups, and can assist organizations in making strategic decisions. Progress in information technology has made it possible to acquire more and more data about human behavior. In this paper, we examine behavior data obtained in real-world scenarios as an information network composed of two types of objects (humans and actions) associated with various attributes and three types of relationships (human-human, human-action, and action-action), which we call the heterogeneous behavior network (HBN). To exploit the abundance and heterogeneity of the HBN, we propose a novel network embedding method, human-action-attribute-aware heterogeneous network embedding (a4HNE), which jointly considers structural proximity, attribute resemblance, and heterogeneity fusion. Experiments on two real-world datasets show that this approach outperforms other similar methods on various heterogeneous information network mining tasks for potential behavior prediction.

Network representation learning: models, methods and applications

With the rise of large-scale social networks, network mining has become an important sub-domain of data mining. Generating an efficient network representation is one important challenge in applying machine learning to network data. Recently, representation learning methods are widely used in various domains to generate low dimensional latent features from complex high dimensional data. A significant amount of research effort is made in the past few years to generate node representations from graph-structured data using representation learning methods. Here, we provide a detailed study of the latest advancements in the field of network representation learning (also called network embedding). We first discuss the basic concepts and models of network embedding. Further, we build a taxonomy of network embedding methods based on the type of networks and review the major research works that come under each category. We then cover the major datasets used in network embedding research and describe the major applications of network embedding with respect to various network mining tasks. Finally, we provide various directions for future work which enhance further research.

Graph representation learning with encoding edges

Network embedding aims at learning the low dimensional representation of nodes. These representations can be widely used for network mining tasks, such as link prediction, anomaly detection, and classification. Recently, a great deal of meaningful research work has been carried out on this emerging network analysis paradigm. The real-world network contains different size clusters because of the edges with different relationship types. These clusters also reflect some features of nodes, which can contribute to the optimization of the feature representation of nodes. However, existing network embedding methods do not distinguish these relationship types. In this paper, we propose an unsupervised network representation learning model that can encode edge relationship information. Firstly, an objective function is defined, which can learn the edge vectors by implicit clustering. Then, a biased random walk is designed to generate a series of node sequences, which are put into Skip-Gram to learn the low dimensional node representations. Extensive experiments are conducted on several network datasets. Compared with the state-of-art baselines, the proposed method is able to achieve favorable and stable results in multi-label classification and link prediction tasks.

Outlier Aware Network Embedding for Attributed Networks

Attributed network embedding has received much interest from the research community as most of the networks come with some content in each node, which is also known as node attributes. Existing attributed network approaches work well when the network is consistent in structure and attributes, and nodes behave as expected. But real world networks often have anomalous nodes. Typically these outliers, being relatively unexplainable, affect the embeddings of other nodes in the network. Thus all the downstream network mining tasks fail miserably in the presence of such outliers. Hence an integrated approach to detect anomalies and reduce their overall effect on the network embedding is required.Towards this end, we propose an unsupervised outlier aware network embedding algorithm (ONE) for attributed networks, which minimizes the effect of the outlier nodes, and hence generates robust network embeddings. We align and jointly optimize the loss functions coming from structure and attributes of the network. To the best of our knowledge, this is the first generic network embedding approach which incorporates the effect of outliers for an attributed network without any supervision. We experimented on publicly available real networks and manually planted different types of outliers to check the performance of the proposed algorithm. Results demonstrate the superiority of our approach to detect the network outliers compared to the state-of-the-art approaches. We also consider different downstream machine learning applications on networks to show the efficiency of ONE as a generic network embedding technique. The source code is made available at https://github.com/sambaranban/ONE.