Link Prediction Task Research Articles

Transfer learning based attack detection for wireless communication networks

AbstractWireless communication network attack detection is important in the field of wireless communication network data mining. Transfer learning is a powerful tool to solve the problem of link prediction in unlabeled wireless networks. In the feature mapping process of transfer learning, the loss of source domain information is very serious, which leads to the loss of knowledge in inverse mapping. To obtain more specific and complete information in the source domain, a link prediction method based on the distribution function fitting method of domain orthogonal term family is proposed. The key samples in the source domain are used to construct a family of orthogonal polynomials, and the least square approximation of the distribution function is obtained by using the family of orthogonal polynomials. The feature vector set of the key samples is extracted from the source domain, and the pseudo label is assigned to the target training set, so that the link classifier can complete the supervised link prediction task. In addition, an advanced performance of TLAD is obtained by comparing with other state‐of‐the‐art models in practice.

Graph convolutional autoencoders with co-learning of graph structure and node attributes

Recently, graph representation learning based on autoencoders has received much attention. However, these methods suffer from two limitations. First, most graph autoencoders ignore the reconstruction of either the graph structure or the node attributes, which often leads to a poor latent representation of the graph-structured data. Second, for existing graph autoencoders models, the encoder and decoder are mainly composed of an initial graph convolutional network (GCN) or its variants. These traditional GCN-based graph autoencoders more or less encounter the problem of incomplete filtering, which causes these models to be unstable in practical applications. To address the above issues, this paper proposes the Graph convolutional Autoencoders with co-learning of graph Structure and Node attributes (GASN) based on variational autoencoders. Specifically, the proposed GASN encodes and decodes the node attributes and graph structure comprehensively in the graph-structured data. Furthermore, we design a completely low-pass graph encoder and a high-pass graph decoder. The experimental results on real-world datasets demonstrate that the proposed GASN achieves state-of-the-art performance on node clustering, link prediction, and visualization tasks.

New Insights and Methods For Predicting Face-To-Face Contacts

The prediction of new links in social networks is a challeng- ing task. In this paper, we focus on predicting links in net- works of face-to-face spatial proximity by using information from online social networks, such as co-authorship networks in DBLP, and a number of node level attributes. First, we analyze influence factors for the link prediction task. Then, we propose a novel method that combines information from different networks and node level attributes for the pre- diction task: We introduce an unsupervised link prediction method based on rooted random walks, and show that it out- performs state-of-the-art unsupervised link prediction meth- ods. We present an evaluation using three real-world datasets. Furthermore, we discuss the impact of our results and of the insights we glean in the field of link prediction and human contact behavior.

Representing a Heterogeneous Pharmaceutical Knowledge-Graph with Textual Information.

We deal with a heterogeneous pharmaceutical knowledge-graph containing textual information built from several databases. The knowledge graph is a heterogeneous graph that includes a wide variety of concepts and attributes, some of which are provided in the form of textual pieces of information which have not been targeted in the conventional graph completion tasks. To investigate the utility of textual information for knowledge graph completion, we generate embeddings from textual descriptions given to heterogeneous items, such as drugs and proteins, while learning knowledge graph embeddings. We evaluate the obtained graph embeddings on the link prediction task for knowledge graph completion, which can be used for drug discovery and repurposing. We also compare the results with existing methods and discuss the utility of the textual information.

Improving Knowledge Graph Completion Using Soft Rules and Adversarial Learning

Knowledge graph is a useful resources and tools for describing entities and relationships in natural language processing tasks. However, the existing knowledge graph are incomplete. Therefore, knowledge graph completion technology has become a research hotspot in the field of artificial intelligence, but the traditional knowledge graph embedding method does not fully take into account the role of logic rules and the effect of false negative samples on knowledge embedding. Based on the logic rules of knowledge and the role of adversarial learning in knowledge embedding, we proposes a model to improve the completion of knowledge graph: soft Rules and graph adversarial learning (RUGA). Firstly, the traditional knowledge graph embedding model is trained as generator and discriminator by using adversarial learning method, and high-quality negative samples are obtained. Then these negative samples and the existing positive samples together constitute the label triple in the injection rule model. The whole model will benefit from both high-quality samples and logical rules. In addition, we evaluated the performance of link prediction task and triple classification task on Freebase and Yago datasets respectively. Finally, the experimental results show that the model can effectively improve the effect of knowledge graph completion.

Directed closure coefficient and its patterns.

The triangle structure, being a fundamental and significant element, underlies many theories and techniques in studying complex networks. The formation of triangles is typically measured by the clustering coefficient, in which the focal node is the centre-node in an open triad. In contrast, the recently proposed closure coefficient measures triangle formation from an end-node perspective and has been proven to be a useful feature in network analysis. Here, we extend it by proposing the directed closure coefficient that measures the formation of directed triangles. By distinguishing the direction of the closing edge in building triangles, we further introduce the source closure coefficient and the target closure coefficient. Then, by categorising particular types of directed triangles (e.g., head-of-path), we propose four closure patterns. Through multiple experiments on 24 directed networks from six domains, we demonstrate that at network-level, the four closure patterns are distinctive features in classifying network types, while at node-level, adding the source and target closure coefficients leads to significant improvement in link prediction task in most types of directed networks.

An information theoretic approach to link prediction in multiplex networks

The entities of real-world networks are connected via different types of connections (i.e., layers). The task of link prediction in multiplex networks is about finding missing connections based on both intra-layer and inter-layer correlations. Our observations confirm that in a wide range of real-world multiplex networks, from social to biological and technological, a positive correlation exists between connection probability in one layer and similarity in other layers. Accordingly, a similarity-based automatic general-purpose multiplex link prediction method—SimBins—is devised that quantifies the amount of connection uncertainty based on observed inter-layer correlations in a multiplex network. Moreover, SimBins enhances the prediction quality in the target layer by incorporating the effect of link overlap across layers. Applying SimBins to various datasets from diverse domains, our findings indicate that SimBins outperforms the compared methods (both baseline and state-of-the-art methods) in most instances when predicting links. Furthermore, it is discussed that SimBins imposes minor computational overhead to the base similarity measures making it a potentially fast method, suitable for large-scale multiplex networks.

A generative adversarial network for single and multi-hop distributional knowledge base completion

Knowledge bases (KBs) inherently lack reasoning ability, limiting their effectiveness for tasks such as question–answering and query expansion. Machine-learning is hence commonly employed for representation learning in order to learn semantic features useful for generalization. Most existing methods utilize discriminative models that require both positive and negative samples to learn a decision boundary. KBs, by contrast, contain only positive samples, necessitating that negative samples are generated by replacing the head/tail of predicates with randomly-chosen entities. They are thus frequently easily discriminable from positive samples, which can prevent learning of sufficiently robust classifiers.Generative models, however, do not require negative samples to learn the distribution of positive samples; stimulated by recent developments in Generative Adversarial Networks (GANs), we propose a novel framework, Knowledge Completion GANs (KCGANs), for competitively training generative link prediction models against discriminative belief prediction models. KCGAN thus invokes a game between generator-network G and discriminator-network D in which G aims to understand underlying KB structure by learning to perform link prediction while D tries to gain knowledge about the KB by learning predicate/triplet classification. Two key challenges are addressed: 1) Classical GAN architectures’ inability to easily generate samples over discrete entities; 2) the inefficiency of softmax for learning distributions over large sets of entities. As a step toward full first-order logical reasoning we further extend KCGAN to learn multi-hop logical entailment relations between entities by enabling G to compose a multi-hop relational path between entities and D to discriminate between real and fake paths.KCGAN is tested on benchmarks WordNet and FreeBase datasets and evaluated on link prediction and belief prediction tasks using MRR and HIT@ 10, achieving best-in-class performance.

MixSp: A Framework for Embedding Heterogeneous Information Networks With Arbitrary Number of Node and Edge Types

Heterogeneous information network (HIN) embedding is to encode network structure into node representations with the heterogeneous semantics of different node and edge types considered. However, since each HIN may have a unique nature, e.g., a unique set of node and edge types, a model designed for one type of networks may not be applicable to or effective on another type. In this article, we thus attempt to propose a framework for HINs with arbitrary number of node and edge types. The proposed framework constructs a novel mixture-split representation of an HIN, and hence is named as MixSp. The mixture sub-representation and the split sub-representation serve as two different views of the network. Compared with existing models which only learn from the original view, MixSp thus may exploit more comprehensive information. Node representations in each view are learned by embedding the respective network structure. Moreover, the node representations are further refined through cross-view co-regularization. The framework is instantiated in three models which differ from each other in the co-regularization. Extensive experiments on three real-world datasets show MixSp outperforms several recent models in both node classification and link prediction tasks even though MixSp is not designed for a particular type of HINs.

Hierarchical-aware relation rotational knowledge graph embedding for link prediction

Knowledge graph embedding, as the upstream task of link prediction which aims to predict new links between entities under the premise of known relations, its reliability greatly affects the performance of link prediction. However, previous distance-based models focus on modeling complicated relation patterns while ignoring the semantic hierarchy of knowledge graph, from TransE to RotatE. In this setting, all entities are regarded as the same type, and the fact that different entities belong to different levels is neglected. Therefore, we propose the general form of RotatE, the hierarchical-aware relation rotational knowledge graph embedding (HA-RotatE), to model the hierarchical-aware knowledge graph. HA-RotatE represents entities and relations as complex vectors and uses different moduli of entity embeddings to indicate the different hierarchical levels they belong to. The transformation of modulus and rotation from head entity to tail entity depends on different relations. Some relations are used to link entities of the same level, and others are used to link entities of different levels. We adopt the shared modulus transformation parameter method for avoiding overfitting. As the general form of RotatE, HA-RotatE also has the ability to model and infer various relation modes, i.e., symmetry/antisymmetric, inversion and composition. On benchmark datasets WN18RR and FB15k-237, the experiments on link prediction tasks show that: (1) HA-RotatE can effectively model the semantic hierarchy of the knowledge graph; (2) Compared with competitive benchmarks, our model substantially outperforms them in most metrics.

Location-Sensitive Embedding for Knowledge Graph Embedding

<p indent=0mm>Knowledge graph embedding maps the discrete symbolic entities and relations into continuous low-dimensional space, enabling the downstream task (e.g., inference, knowledge graph completion and etc.) to be carried on in an algebraic manner. Roughly speaking, there are two main-streams towards knowledge graph embedding, translational distance models and semantic matching models. The learning bottleneck of the former caused by special graph structures is analyzed. And it is found out that translational distance models are to be blamed for treating head and tail entities from a same viewpoint. So, a location-sensitive embedding (LSE) model is proposed. Unlike previous models, it just transforms the head entity with relational-specific mapping. And it models relation as a general linear transformation instead of a translational operator. Its representation capacity and the relationship to current models are theoretically analyzed. Also, to make it efficient, the model is simplified by assuming a diagonal matrix for transformation, forming a simplified model LSE<italic>_d</italic>. It is evaluated that the proposed model on four large-scale knowledge graph datasets for link prediction task. Experiments have shown that the proposed model achieves highest, or at least competitive, performance compared with the state-of-the-art models.

Generalized Relation Learning with Semantic Correlation Awareness for Link Prediction

Developing link prediction models to automatically complete knowledge graphs has recently been the focus of significant research interest. The current methods for the link prediction task have two natural problems: 1) the relation distributions in KGs are usually unbalanced, and 2) there are many unseen relations that occur in practical situations. These two problems limit the training effectiveness and practical applications of the existing link prediction models. We advocate a holistic understanding of KGs and we propose in this work a unified Generalized Relation Learning framework GRL to address the above two problems, which can be plugged into existing link prediction models. GRL conducts a generalized relation learning, which is aware of semantic correlations between relations that serve as a bridge to connect semantically similar relations. After training with GRL, the closeness of semantically similar relations in vector space and the discrimination of dissimilar relations are improved. We perform comprehensive experiments on six benchmarks to demonstrate the superior capability of GRL in the link prediction task. In particular, GRL is found to enhance the existing link prediction models making them insensitive to unbalanced relation distributions and capable of learning unseen relations.

ChronoR: Rotation Based Temporal Knowledge Graph Embedding

Despite the importance and abundance of temporal knowledge graphs, most of the current research has been focused on reasoning on static graphs. In this paper, we study the challenging problem of inference over temporal knowledge graphs. In particular, the task of temporal link prediction. In general, this is a difficult task due to data non-stationarity, data heterogeneity, and its complex temporal dependencies. We propose Chronological Rotation embedding (ChronoR), a novel model for learning representations for entities, relations, and time. Learning dense representations is frequently used as an efficient and versatile method to perform reasoning on knowledge graphs. The proposed model learns a k-dimensional rotation transformation parametrized by relation and time, such that after each fact's head entity is transformed using the rotation, it falls near its corresponding tail entity. By using high dimensional rotation as its transformation operator, ChronoR captures rich interaction between the temporal and multi-relational characteristics of a Temporal Knowledge Graph. Experimentally, we show that ChronoR is able to outperform many of the state-of-the-art methods on the benchmark datasets for temporal knowledge graph link prediction.

Dual Quaternion Knowledge Graph Embeddings

In this paper, we study the problem of learning representations of entities and relations in the knowledge graph for the link prediction task. Our idea is based on the observation that the vast majority of the related work only models the relation as a single geometric operation such as translation or rotation, which limits the representation power of the underlying models and makes it harder to match the complicated relations existed in real-world datasets. To embrace a richer set of relational information, we propose a new method called dual quaternion knowledge graph embedding (DualE), which introduces dual quaternions into knowledge graph embeddings. Specifically, a dual quaternion behaves like a “complex quaternion” with its real and imaginary part all being quaternary. The core of DualE lies a specific design of dual-quaternion-based multiplication, which universally models relations as the compositions of a series of translation and rotation operations. The major merits of DualE are three-fold:1) it is the first unified framework embracing both rotation based and translation-based models, 2) it expands the embedding space to the dual quaternion space with a more intuitive physical and geometric interpretation, 3) it satisfies the key patterns and the multiple relations pattern of relational representation learning. Experimental results on four real-world datasets demonstrate the effectiveness of our DualE method.

Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs

Inductive link prediction---where entities during training and inference stages can be different---has been shown to be promising for completing continuously evolving knowledge graphs. Existing models of inductive reasoning mainly focus on predicting missing links by learning logical rules. However, many existing approaches do not take into account semantic correlations between relations, which are commonly seen in real-world knowledge graphs. To address this challenge, we propose a novel inductive reasoning approach, namely TACT, which can effectively exploit Topology-Aware CorrelaTions between relations in an entity-independent manner. TACT is inspired by the observation that the semantic correlation between two relations is highly correlated to their topological structure in knowledge graphs. Specifically, we categorize all relation pairs into several topological patterns, and then propose a Relational Correlation Network (RCN) to learn the importance of the different patterns for inductive link prediction. Experiments demonstrate that TACT can effectively model semantic correlations between relations, and significantly outperforms existing state-of-the-art methods on benchmark datasets for the inductive link prediction task.

Identity-aware Graph Neural Networks

Message passing Graph Neural Networks (GNNs) provide a powerful modeling framework for relational data. However, the expressive power of existing GNNs is upper-bounded by the 1-Weisfeiler-Lehman (1-WL) graph isomorphism test, which means GNNs that are not able to predict node clustering coefficients and shortest path distances, and cannot differentiate between different d-regular graphs. Here we develop a class of message passing GNNs, named Identity-aware Graph Neural Networks (ID-GNNs), with greater expressive power than the 1-WL test. ID-GNN offers a minimal but powerful solution to limitations of existing GNNs. ID-GNN extends existing GNN architectures by inductively considering nodes’ identities during message passing. To embed a given node, ID-GNN first extracts the ego network centered at the node, then conducts rounds of heterogeneous message passing, where different sets of parameters are applied to the center node than to other surrounding nodes in the ego network. We further propose a simplified but faster version of ID-GNN that injects node identity information as augmented node features. Alto- gether, both versions of ID-GNN represent general extensions of message passing GNNs, where experiments show that transforming existing GNNs to ID-GNNs yields on average 40% accuracy improvement on challenging node, edge, and graph property prediction tasks; 3% accuracy improvement on node and graph classification benchmarks; and 15% ROC AUC improvement on real-world link prediction tasks. Additionally, ID-GNNs demonstrate improved or comparable performance over other task-specific graph networks.

Relational Learning Analysis of Social Politics using Knowledge Graph Embedding

Knowledge Graphs (KGs) have gained considerable attention recently from both academia and industry. In fact, incorporating graph technology and the copious of various graph datasets have led the research community to build sophisticated graph analytics tools, which has extended the application of KGs to tackle a plethora of real-life problems in dissimilar domains. Despite the abundance of the currently proliferated generic KGs, there is a vital need to construct domain-specific KGs. Further, quality and credibility should be assimilated in the process of constructing and augmenting KGs, particularly those propagated from mixed-quality resources such as social media data. For example, the amount of the political discourses in social media is overwhelming yet can be hijacked and misused by spammers to spread misinformation and false news. This paper presents a novel credibility domain-based KG Embedding framework. This framework involves capturing a fusion of data related to politics domain and obtained from heterogeneous resources into a formal KG representation depicted by a politics domain ontology. The proposed approach makes use of various knowledge-based repositories to enrich the semantics of the textual contents, thereby facilitating the interoperability of information. The proposed framework also embodies a domain-based social credibility module to ensure data quality and trustworthiness. The utility of the proposed framework is verified by means of experiments conducted on two constructed KGs. The KGs are then embedded in low-dimensional semantically-continuous space using several embedding techniques. The effectiveness of embedding techniques and social credibility module is further demonstrated and substantiated on link prediction, clustering, and visualisation tasks.

SRGCN: Graph-based multi-hop reasoning on knowledge graphs

Learning to infer missing links is one of the fundamental tasks in the knowledge graph. Instead of reasoning based on separate paths in the existing methods, in this paper, we propose a new model, Sequential Relational Graph Convolutional Network (SRGCN), which treats the multiple paths between an entity pair as a sequence of subgraphs. Specifically, to reason the relationship between two entities, we first construct a graph for the entities based on the knowledge graph and serialize the graph to a sequence. For each hop in the sequence, Relational Graph Convolutional Network (R-GCN) is then applied to update the embeddings of the entities. The updated embedding of the tail entity contains information of the entire graph, hence the relationship between two entities can be inferred from it. Compared to the existing approaches that deal with paths separately, SRGCN treats the graph as a whole, which can encode structural information and interactions between paths better. Experiments show that SRGCN outperforms path-based baselines on both link and fact prediction tasks. We also show that SRGCN is highly efficient in the sense that only one epoch of training is enough to achieve high accuracy, and even partial datasets can lead to competitive performance.

Relation-based multi-type aware knowledge graph embedding

Knowledge graph (KG) embedding projects the graph into a low-dimensional space and preserves the graph information. An essential part of a KG is the ontology, which always is organized as a taxonomy tree, depicting the type (or multiple types) of each entity and the hierarchical relationships among these types. The importance of considering the ontology during KG embedding lies in its ability to provide side-information, improving the downstream applications’ accuracy (e.g., link prediction, entity alignment or recommendation). However, the ontology has yet to receive adequate attention during the KG embedding, especially for instances where each entity may belong to multiple types. This ontology-enhanced KG embedding’s main challenges are twofold: determining how to discover the relationships among these types and how to integrate them with the entities’ relationship network. Although it is common to see attention-based models used in KG embedding, they cannot settle the issues raised simultaneously. Only a single type is assigned to each entity and the correlation among types are ignored in those models, leading to information loss and encumbered downstream tasks. To overcome these challenges, we propose a composite multi-type aware KG embedding model, whose main components are a multi-type layer and entity embedding layer. We model it as a natural language processing task at the multi-type layer to discover each entity’s multi-type feature and automatically capture their correlations. Additionally, a relation-based attention mechanism is conducted at the entity embedding layer, which aggregates neighborhoods’ information and integrates the multi-type layer’s information through common entities of these two layers. Through extensive experiments on two real KGs, we demonstrate that, compared to several state-of-the-art baselines, our Multi-Type aware Embedding (MTE) model achieves substantial gain in both Mean Rank and Hit@N for the link prediction task and accuracy for multi-type classification.

W-MMP2Vec: Topic-driven network embedding model for link prediction in content-based heterogeneous information network

Link prediction on heterogeneous information network (HIN) is considered as a challenge problem due to the complexity and diversity in types of nodes and links. Currently, there are remained challenges of meta-path-based link prediction in HIN. Previous works of link prediction in HIN via network embedding approach are mainly focused on exploiting features of node rather than existing relations in forms of meta-paths between nodes. In fact, predicting the existence of new links between non-linked nodes is absolutely inconvincible. Moreover, recent HIN-based embedding models also lack of thorough evaluations on the topic similarity between text-based nodes along given meta-paths. To tackle these challenges, in this paper, we proposed a novel approach of topic-driven multiple meta-path-based HIN representation learning framework, namely W-MMP2Vec. Our model leverages the quality of node representations by combining multiple meta-paths as well as calculating the topic similarity weight for each meta-path during the processes of network embedding learning in content-based HINs. To validate our approach, we apply W-TMP2Vec model in solving several link prediction tasks in both content-based and non-content-based HINs (DBLP, IMDB and BlogCatalog). The experimental outputs demonstrate the effectiveness of proposed model which outperforms recent state-of-the-art HIN representation learning models.