Graph Embedding Techniques Research Articles

Learning knowledge graph embedding with a bi-directional relation encoding network and a convolutional autoencoder decoding network

Derived from knowledge bases, knowledge graphs represent knowledge expressions in graphs, which utilize nodes and edges to denote entities and relations conceptually. Knowledge graph can be described in textual triple form, consisting of head entities, tail entities and relations between entities. In order to represent elements in knowledge graphs, knowledge graph embedding techniques are proposed to map entities and relations into continuous vector spaces as numeric vectors for computational efficiency. Convolution-based knowledge graph embedding models have promising performance for knowledge graph representation learning. However, the input of those neural network-based models is frequently in handmade forms and may suffer from low efficiency in feature extraction procedure of the models. In this paper, a convolutional autoencoder is proposed for knowledge graph representation learning with entity pairs as input, aiming to obtain corresponding hidden relation representation. In addition, a bi-directional relation encoding network is utilized to represent semantic of entities in different directional relation patterns, as an encoder to output representation for initialization of the convolutional autoencoder. Experiments are conducted on standard datasets including, WN18RR, Kinship, NELL-995 and FB15k-237 as a link prediction task. Besides, input embedding matrix composed of different ingredients is designed to evaluate performances of the convolutional autoencoder. The results demonstrate that our model is effective in learning representation from entity feature interactions.

A Scalable and Efficient Multi-Agent Architecture for Malware Protection in Data Sharing Over Mobile Cloud

Sensitive Data requires encryption before uploading to a public cloud. Access control based on Attribute-Based Encryption (ABE) is an effective technique to ensure data shared security and privacy in the public cloud. Cipher-Text Policy of ABE may suffer from scalability and performance issues as they do not permit for addition or removal of computing nodes at run time. Furthermore, another problem is existing approaches suffer from single-point-of-failure (SPoF). Therefore, we introduce a scalable multi-agent system architecture based on CP-ABE to ensure data sharing on public cloud storage and reliability in our proposed work. We proposed a cloud host as an inter-mediator between the user and the authorized agents without violating the system's privacy and security. We have also proposed a novel methodology to protect the cloud from malware by exploiting the state's efficient power of the art Gemini approach. Gemini is an efficient methodology for binary code-based graph embedding similarity detection. Our proposed study overcomes the deficiencies of scalability and efficiency along with providing the mechanism for malware detection in the cloud. It covers three aspects: scalability, the efficiency with multi-agents, and malware detection capability. Our contributed work is scalable, efficient for cloud data sharing, and protects from malware. Results reveal that our work provides better performance with preserving the security, privacy, and fine granularity features of CP-ABE and malware screening using regress analysis by the graph embedding technique.

Exploring chromatin conformation and gene co-expression through graph embedding.

The relationship between gene co-expression and chromatin conformation is of great biological interest. Thanks to high-throughput chromosome conformation capture technologies (Hi-C), researchers are gaining insights on the tri-dimensional organization of the genome. Given the high complexity of Hi-C data and the difficult definition of gene co-expression networks, the development of proper computational tools to investigate such relationship is rapidly gaining the interest of researchers. One of the most fascinating questions in this context is how chromatin topology correlates with gene co-expression and which physical interaction patterns are most predictive of co-expression relationships. To address these questions, we developed a computational framework for the prediction of co-expression networks from chromatin conformation data. We first define a gene chromatin interaction network where each gene is associated to its physical interaction profile; then, we apply two graph embedding techniques to extract a low-dimensional vector representation of each gene from the interaction network; finally, we train a classifier on gene embedding pairs to predict if they are co-expressed. Both graph embedding techniques outperform previous methods based on manually designed topological features, highlighting the need for more advanced strategies to encode chromatin information. We also establish that the most recent technique, based on random walks, is superior. Overall, our results demonstrate that chromatin conformation and gene regulation share a non-linear relationship and that gene topological embeddings encode relevant information, which could be used also for downstream analysis. The source code for the analysis is available at: https://github.com/marcovarrone/gene-expression-chromatin. Supplementary data are available at Bioinformatics online.

Enhancing fine-grained intra-urban dengue forecasting by integrating spatial interactions of human movements between urban regions.

BackgroundAs a mosquito-borne infectious disease, dengue fever (DF) has spread through tropical and subtropical regions worldwide in recent decades. Dengue forecasting is essential for enhancing the effectiveness of preventive measures. Current studies have been primarily conducted at national, sub-national, and city levels, while an intra-urban dengue forecasting at a fine spatial resolution still remains a challenging feat. As viruses spread rapidly because of a highly dynamic population flow, integrating spatial interactions of human movements between regions would be potentially beneficial for intra-urban dengue forecasting.MethodologyIn this study, a new framework for enhancing intra-urban dengue forecasting was developed by integrating the spatial interactions between urban regions. First, a graph-embedding technique called Node2Vec was employed to learn the embeddings (in the form of an N-dimensional real-valued vector) of the regions from their population flow network. As strongly interacting regions would have more similar embeddings, the embeddings can serve as “interaction features.” Then, the interaction features were combined with those commonly used features (e.g., temperature, rainfall, and population) to enhance the supervised learning–based dengue forecasting models at a fine-grained intra-urban scale.ResultsThe performance of forecasting models (i.e., SVM, LASSO, and ANN) integrated with and without interaction features was tested and compared on township-level dengue forecasting in Guangzhou, the most threatened sub-tropical city in China. Results showed that models using both common and interaction features can achieve better performance than that using common features alone.ConclusionsThe proposed approach for incorporating spatial interactions of human movements using graph-embedding technique is effective, which can help enhance fine-grained intra-urban dengue forecasting.

Exploiting non-taxonomic relations for measuring semantic similarity and relatedness in WordNet

Various applications in computational linguistics and artificial intelligence employ semantic similarity to solve challenging tasks, such as word sense disambiguation, text classification, information retrieval, machine translation, and document clustering. To our knowledge, research to date rely solely on the taxonomic relation “ISA” to evaluate semantic similarity and relatedness between terms. This paper explores the benefits of using all types of non-taxonomic relations in large linked data, such as WordNet knowledge graph, to enhance existing semantic similarity and relatedness measures. We propose a holistic poly-relational approach based on a new relation-based information content and non-taxonomic-based weighted paths to devise a comprehensive semantic similarity and relatedness measure. To demonstrate the benefits of exploiting non-taxonomic relations in a knowledge graph, we used three strategies to deploy non-taxonomic relations at different granularity levels. We conduct experiments on four well-known gold standard datasets. The results of our proposed method demonstrate an improvement over the benchmark semantic similarity methods, including the state-of-the-art knowledge graph embedding techniques, that ranged from 3.8%–23.8%, 1.3%–18.3%, 31.8%–117.2%, and 19.1%–111.1%, on all gold standard datasets MC, RG, WordSim, and Mturk, respectively. These results demonstrate the robustness and scalability of the proposed semantic similarity and relatedness measure, significantly improving existing similarity measures.

Approximating Network Centrality Measures Using Node Embedding and Machine Learning

Extracting information from real-world large networks is a key challenge nowadays. For instance, computing a node centrality may become unfeasible depending on the intended centrality due to its computational cost. One solution is to develop fast methods capable of approximating network centralities. Here, we propose an approach for efficiently approximating node centralities for large networks using Neural Networks and Graph Embedding techniques. Our proposed model, entitled Network Centrality Approximation using Graph Embedding (NCA-GE), uses the adjacency matrix of a graph and a set of features for each node (here, we use only the degree) as input and computes the approximate desired centrality rank for every node. NCA-GE has a time complexity of O(|E|), E being the set of edges of a graph, making it suitable for large networks. NCA-GE also trains pretty fast, requiring only a set of a thousand small synthetic scale-free graphs (ranging from 100 to 1000 nodes each), and it works well for different node centralities, network sizes, and topologies. Finally, we compare our approach to the state-of-the-art method that approximates centrality ranks using the degree and eigenvector centralities as input, where we show that the NCA-GE outperforms the former in a variety of scenarios.

Use of word and graph embedding to measure semantic relatedness between Unified Medical Language System concepts.

ObjectiveThe study sought to explore the use of deep learning techniques to measure the semantic relatedness between Unified Medical Language System (UMLS) concepts.Materials and MethodsConcept sentence embeddings were generated for UMLS concepts by applying the word embedding models BioWordVec and various flavors of BERT to concept sentences formed by concatenating UMLS terms. Graph embeddings were generated by the graph convolutional networks and 4 knowledge graph embedding models, using graphs built from UMLS hierarchical relations. Semantic relatedness was measured by the cosine between the concepts’ embedding vectors. Performance was compared with 2 traditional path-based (shortest path and Leacock-Chodorow) measurements and the publicly available concept embeddings, cui2vec, generated from large biomedical corpora. The concept sentence embeddings were also evaluated on a word sense disambiguation (WSD) task. Reference standards used included the semantic relatedness and semantic similarity datasets from the University of Minnesota, concept pairs generated from the Standardized MedDRA Queries and the MeSH (Medical Subject Headings) WSD corpus.ResultsSentence embeddings generated by BioWordVec outperformed all other methods used individually in semantic relatedness measurements. Graph convolutional network graph embedding uniformly outperformed path-based measurements and was better than some word embeddings for the Standardized MedDRA Queries dataset. When used together, combined word and graph embedding achieved the best performance in all datasets. For WSD, the enhanced versions of BERT outperformed BioWordVec.ConclusionsWord and graph embedding techniques can be used to harness terms and relations in the UMLS to measure semantic relatedness between concepts. Concept sentence embedding outperforms path-based measurements and cui2vec, and can be further enhanced by combining with graph embedding.

Characterizing the Complexity of Weighted Networks via Graph Embedding and Point Pattern Analysis.

We propose a new metric to characterize the complexity of weighted complex networks. Weighted complex networks represent a highly organized interactive process, for example, co-varying returns between stocks (financial networks) and coordination between brain regions (brain connectivity networks). Although network entropy methods have been developed for binary networks, the measurement of non-randomness and complexity for large weighted networks remains challenging. We develop a new analytical framework to measure the complexity of a weighted network via graph embedding and point pattern analysis techniques in order to address this unmet need. We first perform graph embedding to project all nodes of the weighted adjacency matrix to a low dimensional vector space. Next, we analyze the point distribution pattern in the projected space, and measure its deviation from the complete spatial randomness. We evaluate our method via extensive simulation studies and find that our method can sensitively detect the difference of complexity and is robust to noise. Last, we apply the approach to a functional magnetic resonance imaging study and compare the complexity metrics of functional brain connectivity networks from 124 patients with schizophrenia and 103 healthy controls. The results show that the brain circuitry is more organized in healthy controls than schizophrenic patients for male subjects while the difference is minimal in female subjects. These findings are well aligned with the established sex difference in schizophrenia.

Credible seed identification for large-scale structural network alignment

Structural network alignment utilizes the topological structure information to find correspondences between nodes of two networks. Researchers have proposed a line of useful algorithms which usually require a prior mapping of seeds acting as landmark points to align the rest nodes. Several seed-free algorithms are developed to solve the cold-start problem. However, existing approaches suffer high computational cost and low reliability, limiting their applications to large-scale network alignment. Moreover, there is a lack of useful metrics to quantify the credibility of seed mappings. To address these issues, we propose a credible seed identification framework and develop a metric to assess the reliability of a mapping. To tackle the cold-start problem, we employ graph embedding techniques to represent nodes by structural feature vectors in a latent space. We then leverage point set registration algorithms to match nodes algebraically and obtain an initial mapping of nodes. Besides, we propose a heuristic algorithm to improve the credibility of the initial mapping by filtering out mismatched node pairs. To tackle the computational problem in large-scale network alignment, we propose a divide-and-conquer scheme to divide large networks into smaller ones and then match them individually. It significantly improves the recall of mapping results. Finally, we conduct extensive experiments to evaluate the effectiveness and efficiency of our new approach. The results illustrate that the proposed method outperforms the state-of-the-art approaches in terms of both effectiveness and efficiency.

T-EDGE: Temporal WEighted MultiDiGraph Embedding for Ethereum Transaction Network Analysis

Recently, graph embedding techniques have been widely used in the analysis of various networks, but most of the existing embedding methods omit the network dynamics and the multiplicity of edges, so it is difficult to accurately describe the detailed characteristics of the transaction networks. Ethereum is a blockchain-based platform supporting smart contracts. The open nature of blockchain makes the transaction data on Ethereum completely public, and also brings unprecedented opportunities for the transaction network analysis. By taking the realistic rules and features of transaction networks into consideration, we first model the Ethereum transaction network as a Temporal Weighted Multidigraph (TWMDG), where each node is a unique Ethereum account and each edge represents a transaction weighted by amount and assigned with timestamp. Then we define the problem of Temporal Weighted Multidigraph Embedding (T-EDGE) by incorporating both temporal and weighted information of the edges, the purpose being to capture more comprehensive properties of dynamic transaction networks. To evaluate the effectiveness of the proposed embedding method, we conduct experiments of node classification on real-world transaction data collected from Ethereum. Experimental results demonstrate that T-EDGE outperforms baseline embedding methods, indicating that time-dependent walks and multiplicity characteristic of edges are informative and essential for time-sensitive transaction networks.

DTiGEMS+: drug\u2013target interaction prediction using graph embedding, graph mining, and similarity-based techniques

In silico prediction of drug–target interactions is a critical phase in the sustainable drug development process, especially when the research focus is to capitalize on the repositioning of existing drugs. However, developing such computational methods is not an easy task, but is much needed, as current methods that predict potential drug–target interactions suffer from high false-positive rates. Here we introduce DTiGEMS+, a computational method that predicts Drug–Target interactions using Graph Embedding, graph Mining, and Similarity-based techniques. DTiGEMS+ combines similarity-based as well as feature-based approaches, and models the identification of novel drug–target interactions as a link prediction problem in a heterogeneous network. DTiGEMS+ constructs the heterogeneous network by augmenting the known drug–target interactions graph with two other complementary graphs namely: drug–drug similarity, target–target similarity. DTiGEMS+ combines different computational techniques to provide the final drug target prediction, these techniques include graph embeddings, graph mining, and machine learning. DTiGEMS+ integrates multiple drug–drug similarities and target–target similarities into the final heterogeneous graph construction after applying a similarity selection procedure as well as a similarity fusion algorithm. Using four benchmark datasets, we show DTiGEMS+ substantially improves prediction performance compared to other state-of-the-art in silico methods developed to predict of drug-target interactions by achieving the highest average AUPR across all datasets (0.92), which reduces the error rate by 33.3% relative to the second-best performing model in the state-of-the-art methods comparison.

FoodKG: A Tool to Enrich Knowledge Graphs Using Machine Learning Techniques.

While there exist a plethora of datasets on the Internet related to Food, Energy, and Water (FEW), there is a real lack of reliable methods and tools that can consume these resources. This hinders the development of novel decision-making applications utilizing knowledge graphs. In this paper, we introduce a novel software tool, called FoodKG, that enriches FEW knowledge graphs using advanced machine learning techniques. Our overarching goal is to improve decision-making and knowledge discovery as well as to provide improved search results for data scientists in the FEW domains. Given an input knowledge graph (constructed on raw FEW datasets), FoodKG enriches it with semantically related triples, relations, and images based on the original dataset terms and classes. FoodKG employs an existing graph embedding technique trained on a controlled vocabulary called AGROVOC, which is published by the Food and Agriculture Organization of the United Nations. AGROVOC includes terms and classes in the agriculture and food domains. As a result, FoodKG can enhance knowledge graphs with semantic similarity scores and relations between different classes, classify the existing entities, and allow FEW experts and researchers to use scientific terms for describing FEW concepts. The resulting model obtained after training on AGROVOC was evaluated against the state-of-the-art word embedding and knowledge graph embedding models that were trained on the same dataset. We observed that this model outperformed its competitors based on the Spearman Correlation Coefficient score.

Learning Triple Embeddings from Knowledge Graphs

Graph embedding techniques allow to learn high-quality feature vectors from graph structures and are useful in a variety of tasks, from node classification to clustering. Existing approaches have only focused on learning feature vectors for the nodes and predicates in a knowledge graph. To the best of our knowledge, none of them has tackled the problem of directly learning triple embeddings. The approaches that are closer to this task have focused on homogeneous graphs involving only one type of edge and obtain edge embeddings by applying some operation (e.g., average) on the embeddings of the endpoint nodes. The goal of this paper is to introduce Triple2Vec, a new technique to directly embed knowledge graph triples. We leverage the idea of line graph of a graph and extend it to the context of knowledge graphs. We introduce an edge weighting mechanism for the line graph based on semantic proximity. Embeddings are finally generated by adopting the SkipGram model, where sentences are replaced with graph walks. We evaluate our approach on different real-world knowledge graphs and compared it with related work. We also show an application of triple embeddings in the context of user-item recommendations.

Improve the translational distance models for knowledge graph embedding

Knowledge graph embedding techniques can be roughly divided into two mainstream, translational distance models and semantic matching models. Though intuitive, translational distance models fail to deal with the circle structure and hierarchical structure in knowledge graphs. In this paper, we propose a general learning framework named TransX-pa, which takes various models (TransE, TransR, TransH and TransD) into consideration. From this unified viewpoint, we analyse the learning bottlenecks are: (i) the common assumption that the inverse of a relation r is modelled as its opposite − r; and (ii) the failure to capture the rich interactions between entities and relations. Correspondingly, we introduce position-aware embeddings and self-attention blocks, and show that they can be adapted to various translational distance models. Experiments are conducted on different datasets extracted from real-world knowledge graphs Freebase and WordNet in the tasks of both triplet classification and link prediction. The results show that our approach makes a great improvement, showing a better, or comparable, performance with state-of-the-art methods.

Things2Vec: Semantic Modeling in the Internet of Things With Graph Representation Learning

The advent of fifth generation (5G) enables the Internet of Things (IoT) to connect a massive number of things. The interaction and communication among these things generate an enormous amount of context-aware data that is semantically diverse. Traditional data representation approaches, such as semantic annotation, ontology, and semantic Web technology are rule based, which lack flexibility and adaptability when applied to IoT. To address the challenge, this article mainly focuses on the problem of semantic representation, which is essential for processing and fusion of IoT data. To serve as a bridge, we propose a high-level framework, namely, Things2Vec, which aims to produce the latent semantic representations from the interaction of things through the graph embedding technique. These semantic representations benefit various IoT semantic analysis tasks, such as the IoT service recommendation and automation of things. In Things2Vec, we utilize the graph to model the function sequence relationships that are generated by the interaction of things, which is called the IoT context graph. Since these function sequence relationships are heterogeneous in terms of semantics, it causes general graph embeddings to fail to learn complete information. Thus, we propose a biased random walk procedure, which is tailored to capture the neighborhoods of nodes with different types of semantic relationships. Extensive experiments are carried out, and our results show that the proposed method can effectively capture the semantic relationships among context-aware data in IoT.

Risk-Aware Identification of Highly Suspected COVID-19 Cases in Social IoT: A Joint Graph Theory and Reinforcement Learning Approach.

The recent outbreak of the coronavirus disease 2019 (COVID-19) has rapidly become a pandemic, which calls for prompt action in identifying suspected cases at an early stage through risk prediction. To suppress its further spread, we exploit the social relationships between mobile devices in the Social Internet of Things (SIoT) to help control its propagation by allocating the limited protective resources to the influential so-called high-degree individuals to stem the tide of precipitated spreading. By exploiting the so-called differential contact intensity and the infectious rate in susceptible-exposed-infected-removed (SEIR) epidemic model, the resultant optimization problem can be transformed into the minimum weight vertex cover (MWVC) problem of graph theory. To solve this problem in a high-dynamic random network topology, we propose an adaptive scheme by relying on the graph embedding technique during the state representation and reinforcement learning in the training phase. By relying on a pair of real-life datasets, the results demonstrate that our scheme can beneficially reduce the epidemiological reproduction rate of the infection. This technique has the potential of assisting in the early identification of COVID-19 cases.

LBSN2Vec++: Heterogeneous Hypergraph Embedding for Location-Based Social Networks

Location-Based Social Networks (LBSNs) have been widely used as a primary data source for studying the impact of mobility and social relationships on each other. Traditional approaches manually define features to characterize users’ mobility homophily and social proximity, and show that mobility and social features can help friendship and location prediction tasks, respectively. However, these hand-crafted features not only require tedious human efforts, but also are difficult to generalize. Against this background, we propose in this paper LBSN2Vec++, a heterogeneous hypergraph embedding approach designed specifically for LBSN data for automatic feature learning. Specifically, LBSN data intrinsically forms a heterogeneous hypergraph including both user-user homogeneous edges (friendships) and user-time-POI-semantic heterogeneous hyperedges (check-ins). Based on this hypergraph, we first propose a random-walk-with-stay scheme to jointly sample user check-ins and social relationships, and then learn node embeddings from the sampled (hyper)edges by not only preserving the <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> -wise node proximity captured by the hyperedges, but also considering embedding space transformation between node domains to fully grasp the complex structural characteristics of the LBSN heterogeneous hypergraph. Using real-world LBSN datasets collected in six cities all over the world, our extensive evaluation shows that LBSN2Vec++ significantly and consistently outperforms both state-of-the-art graph embedding techniques by up to 68 percent and the best-performing hand-crafted features in the literature by up to 70.14 percent on friendship and location prediction tasks.

Hierarchical stochastic graphlet embedding for graph-based pattern recognition

Despite being very successful within the pattern recognition and machine learning community, graph-based methods are often unusable because of the lack of mathematical operations defined in graph domain. Graph embedding, which maps graphs to a vectorial space, has been proposed as a way to tackle these difficulties enabling the use of standard machine learning techniques. However, it is well known that graph embedding functions usually suffer from the loss of structural information. In this paper, we consider the hierarchical structure of a graph as a way to mitigate this loss of information. The hierarchical structure is constructed by topologically clustering the graph nodes and considering each cluster as a node in the upper hierarchical level. Once this hierarchical structure is constructed, we consider several configurations to define the mapping into a vector space given a classical graph embedding, in particular, we propose to make use of the stochastic graphlet embedding (SGE). Broadly speaking, SGE produces a distribution of uniformly sampled low-to-high-order graphlets as a way to embed graphs into the vector space. In what follows, the coarse-to-fine structure of a graph hierarchy and the statistics fetched by the SGE complements each other and includes important structural information with varied contexts. Altogether, these two techniques substantially cope with the usual information loss involved in graph embedding techniques, obtaining a more robust graph representation. This fact has been corroborated through a detailed experimental evaluation on various benchmark graph datasets, where we outperform the state-of-the-art methods.

Surrogate-Assisted Robust Optimization of Large-Scale Networks Based on Graph Embedding

Robust optimization of complex networks has attracted much attention in recent years. Although existing methods have been successful in achieving promising results, the computational cost for robust optimization tasks is extremely high, which prevents them from being further applied to large-scale networks. Thus, computationally efficient robust optimization methods are in high demand. This article proposes a low-cost method for estimating the robustness of networks with the help of graph embedding techniques and surrogate models. An evolutionary algorithm is then developed to find large-scale robust networks by combining the surrogate-assisted low-cost robustness estimator with the time-consuming real robustness measure by means of a model management strategy. The experimental results on different kinds of synthetic and real networks demonstrate the highly competitive search ability of the proposed algorithm. In addition, the algorithm is able to save up to 80% of the computation time for enhancing the robustness of large-scale networks compared with the state-of-the-art methods.

Domain Adaptation With Neural Embedding Matching

Domain adaptation aims to exploit the supervision knowledge in a source domain for learning prediction models in a target domain. In this article, we propose a novel representation learning-based domain adaptation method, i.e., neural embedding matching (NEM) method, to transfer information from the source domain to the target domain where labeled data is scarce. The proposed approach induces an intermediate common representation space for both domains with a neural network model while matching the embedding of data from the two domains in this common representation space. The embedding matching is based on the fundamental assumptions that a cross-domain pair of instances will be close to each other in the embedding space if they belong to the same class category, and the local geometry property of the data can be maintained in the embedding space. The assumptions are encoded via objectives of metric learning and graph embedding techniques to regularize and learn the semisupervised neural embedding model. We also provide a generalization bound analysis for the proposed domain adaptation method. Meanwhile, a progressive learning strategy is proposed and it improves the generalization ability of the neural network gradually. Experiments are conducted on a number of benchmark data sets and the results demonstrate that the proposed method outperforms several state-of-the-art domain adaptation methods and the progressive learning strategy is promising.