Graph Embedding Learning Research Articles

Estimating Historical Downside Risks of Global Financial Market Indices via Inflation Rate-Adjusted Dependence Graphs

In this study, we examine the application of a graph-based approach to predict the next day's entropic value-at-risk (EVaR) for 21 global financial market indices, focusing on the contemporary issue of high inflation exacerbated by the COVID-19 pandemic. By considering inflation as a conditioning variable, we construct daily networks that delineate interdependencies among key financial indices. Through the application of spatiotemporal graph embedding and machine learning, we predict the EVaR with inflation-adjusted information entropy. Our evaluation demonstrates significant improvements in prediction accuracy using metrics such as root mean squared error (RMSE) and mean absolute error (MAE). Our findings highlight the intricate relationships between inflation rates and downside risks, providing insights into downside risk contagion patterns under high-inflation conditions. This research enhances the understanding of financial market interdependence, the role of inflation in downside risk prediction, and the practicality of network-based analysis, contributing to more stable and refined downside risk predictions within the complex financial landscape during periods of high inflation, which was the different financial environment of the training period.

Deep multi-view graph clustering network with weighting mechanism and collaborative training

With the development of graph convolutional network (GCN), which is powerful in graph embedding learning meanwhile can capture node feature information, deep multi-view graph clustering methods based on graph autoencoder have emerged as a new stream. Although they achieve satisfactory performance, they still have the following weaknesses: (1) some of them model the weights for different views in the encoder part by using attention in the multi-view embedding fusion layer, but fail to consider the weighting in the decoder part to measure the contributions of different views to the reconstruction loss. (2) Most of them directly conduct clustering on the multi-view common embedding layer, but fail to guarantee the alignment of clustering results of different views. To this end, we propose a novel GCN-based deep multi-view graph clustering network with weighting mechanism and collaborative training (DMVGC). The model is composed of multiple view-specific graph encoders and a unified graph decoder. Besides the specially-designed attention module in the encoder part, we construct a reconstruction loss with adaptive weighting mechanism in the decoder part. Additionally, a collaborative self-training clustering objective is jointly conducted on each view-specific embedding layer and the common embedding layer, to make the embedding of each view clustering-friendly toward a common partition. Experiments on several datasets demonstrate the effectiveness of our model.

Graph isomorphism U-Net

Graph embedding learning is a fundamental task when dealing with diverse datasets. While encoder–decoder architectures, such as U-Nets, have shown great success in image pixel-wise prediction tasks, applying similar methods to graph data poses challenges due to the lack of natural pooling and up-sampling operations for graphs. Recent methods leverage learnable parameters to extract structural information from neural networks and extend pooling and unpooling to graphs using node features and graph structural information. This paper proposes a novel model called GIUNet (Graph Isomorphism U-Net) for the graph classification task. The proposed Graph U-Net structure is based on graph isomorphism convolution while using a comprehensive pqPooling layer. The pqPooling layer in our approach effectively combines node features and graph structure information during the graph down-sampling stage. To incorporate graph structure information, we utilize both the spectral representation and node centrality measurements. Node centrality measurements capture various structural aspects of nodes in the graph, while the spectral representation helps us focus on the informative low-frequency components of the graph structure. Through ablation studies, we have demonstrated that leveraging the GIUNet model leads to significant improvements compared to state-of-the-art methods across multiple benchmark datasets.

Multi-View Graph Convolutional Networks with Differentiable Node Selection

Multi-view data containing complementary and consensus information can facilitate representation learning by exploiting the intact integration of multi-view features. Because most objects in the real world often have underlying connections, organizing multi-view data as heterogeneous graphs is beneficial to extracting latent information among different objects. Due to the powerful capability to gather information of neighborhood nodes, in this article, we apply Graph Convolutional Network (GCN) to cope with heterogeneous graph data originating from multi-view data, which is still under-explored in the field of GCN. In order to improve the quality of network topology and alleviate the interference of noises yielded by graph fusion, some methods undertake sorting operations before the graph convolution procedure. These GCN-based methods generally sort and select the most confident neighborhood nodes for each vertex, such as picking the top- k nodes according to pre-defined confidence values. Nonetheless, this is problematic due to the non-differentiable sorting operators and inflexible graph embedding learning, which may result in blocked gradient computations and undesired performance. To cope with these issues, we propose a joint framework dubbed Multi-view Graph Convolutional Network with Differentiable Node Selection (MGCN-DNS), which is constituted of an adaptive graph fusion layer, a graph learning module, and a differentiable node selection schema. MGCN-DNS accepts multi-channel graph-structural data as inputs and aims to learn more robust graph fusion through a differentiable neural network. The effectiveness of the proposed method is verified by rigorous comparisons with considerable state-of-the-art approaches in terms of multi-view semi-supervised classification tasks, and the experimental results indicate that MGCN-DNS achieves pleasurable performance on several benchmark multi-view datasets.

Graph embedding-based link prediction for literature-based discovery in Alzheimer’s Disease

Objective:We explore the framing of literature-based discovery (LBD) as link prediction and graph embedding learning, with Alzheimer’s Disease (AD) as our focus disease context. The key link prediction setting of prediction window length is specifically examined in the context of a time-sliced evaluation methodology. Methods:We propose a four-stage approach to explore literature-based discovery for Alzheimer’s Disease, creating and analyzing a knowledge graph tailored to the AD context, and predicting and evaluating new knowledge based on time-sliced link prediction. The first stage is to collect an AD-specific corpus. The second stage involves constructing an AD knowledge graph with identified AD-specific concepts and relations from the corpus. In the third stage, 20 pairs of training and testing datasets are constructed with the time-slicing methodology. Finally, we infer new knowledge with graph embedding-based link prediction methods. We compare different link prediction methods in this context. The impact of limiting prediction evaluation of LBD models in the context of short-term and longer-term knowledge evolution for Alzheimer’s Disease is assessed. Results:We constructed an AD corpus of over 16 k papers published in 1977–2021, and automatically annotated it with concepts and relations covering 11 AD-specific semantic entity types. The knowledge graph of Alzheimer’s Disease derived from this resource consisted of ∼11 k nodes and ∼394 k edges, among which 34% were genotype-phenotype relationships, 57% were genotype-genotype relationships, and 9% were phenotype-phenotype relationships. A Structural Deep Network Embedding (SDNE) model consistently showed the best performance in terms of returning the most confident set of link predictions as time progresses over 20 years. A huge improvement in model performance was observed when changing the link prediction evaluation setting to consider a more distant future, reflecting the time required for knowledge accumulation. Conclusion:Neural network graph-embedding link prediction methods show promise for the literature-based discovery context, although the prediction setting is extremely challenging, with graph densities of less than 1%. Varying prediction window length on the time-sliced evaluation methodology leads to hugely different results and interpretations of LBD studies. Our approach can be generalized to enable knowledge discovery for other diseases. Availability:Code, AD ontology, and data are available at https://github.com/READ-BioMed/readbiomed-lbd.

Source-Aware Embedding Training on Heterogeneous Information Networks

ABSTRACT Heterogeneous information networks (HINs) have been extensively applied to real-world tasks, such as recommendation systems, social networks, and citation networks. While existing HIN representation learning methods can effectively learn the semantic and structural features in the network, little awareness was given to the distribution discrepancy of subgraphs within a single HIN. However, we find that ignoring such distribution discrepancy among subgraphs from multiple sources would hinder the effectiveness of graph embedding learning algorithms. This motivates us to propose SUMSHINE (Scalable Unsupervised Multi-Source Heterogeneous Information Network Embedding)—a scalable unsupervised framework to align the embedding distributions among multiple sources of an HIN. Experimental results on real-world datasets in a variety of downstream tasks validate the performance of our method over the state-of-the-art heterogeneous information network embedding algorithms.

PromGER: Promoter Prediction Based on Graph Embedding and Ensemble Learning for Eukaryotic Sequence.

Promoters are DNA non-coding regions around the transcription start site and are responsible for regulating the gene transcription process. Due to their key role in gene function and transcriptional activity, the prediction of promoter sequences and their core elements accurately is a crucial research area in bioinformatics. At present, models based on machine learning and deep learning have been developed for promoter prediction. However, these models cannot mine the deeper biological information of promoter sequences and consider the complex relationship among promoter sequences. In this work, we propose a novel prediction model called PromGER to predict eukaryotic promoter sequences. For a promoter sequence, firstly, PromGER utilizes four types of feature-encoding methods to extract local information within promoter sequences. Secondly, according to the potential relationships among promoter sequences, the whole promoter sequences are constructed as a graph. Furthermore, three different scales of graph-embedding methods are applied for obtaining the global feature information more comprehensively in the graph. Finally, combining local features with global features of sequences, PromGER analyzes and predicts promoter sequences through a tree-based ensemble-learning framework. Compared with seven existing methods, PromGER improved the average specificity of 13%, accuracy of 10%, Matthew's correlation coefficient of 16%, precision of 4%, F1 score of 6%, and AUC of 9%. Specifically, this study interpreted the PromGER by the t-distributed stochastic neighbor embedding (t-SNE) method and SHAPley Additive exPlanations (SHAP) value analysis, which demonstrates the interpretability of the model.

Cold-start item recommendation for representation learning based on heterogeneous information networks with fusion side information

In recommendation systems, the cold-start problem refers to the situation that users or items have too few interaction events with the system to be recommended, such as clicks or ratings. The cold-start problem leads to the loss of new users and difficulty in recommending new items and weakens the performance of recommendations. In this paper, we focus on alleviating the cold-start problem of items. Previous work has used the structure of item homogeneous graphs to establish connections between items and generate item embeddings. However, the homogeneous graph of a single node cannot fully explore the potential structural features between users and goods. Due to the flexibility of heterogeneous data, recommendation systems can also employ the Heterogeneous Information Network(HIN) to characterize complex, heterogeneous data. In this paper, a novel Heterogeneous Graph Embedding Learning based Skip-Gram (HSG) model is proposed to mitigate the cold-start problem. First, we introduce three different types of subnetworks and combine them to construct heterogeneous graphs. Then meaningful sequences of nodes are generated via meta-path-based random walk for network embedding. Finally, a Skip-Gram model incorporating side information is used to learn item representations. The learned node embeddings are used as pre-training for the Matrix Factorization (MF) model. The HSG model is proven to be effective by extensive experiments on four publicly available datasets.

A Provable Framework of Learning Graph Embeddings via Summarization

Given a large graph, can we learn its node embeddings from a smaller summary graph? What is the relationship between embeddings learned from original graphs and their summary graphs? Graph representation learning plays an important role in many graph mining applications, but learning em-beddings of large-scale graphs remains a challenge. Recent works try to alleviate it via graph summarization, which typ-ically includes the three steps: reducing the graph size by combining nodes and edges into supernodes and superedges,learning the supernode embedding on the summary graph and then restoring the embeddings of the original nodes. How-ever, the justification behind those steps is still unknown. In this work, we propose GELSUMM, a well-formulated graph embedding learning framework based on graph sum-marization, in which we show the theoretical ground of learn-ing from summary graphs and the restoration with the three well-known graph embedding approaches in a closed form.Through extensive experiments on real-world datasets, we demonstrate that our methods can learn graph embeddings with matching or better performance on downstream tasks.This work provides theoretical analysis for learning node em-beddings via summarization and helps explain and under-stand the mechanism of the existing works.

Health-Aware Food Recommendation Based on Knowledge Graph and Multi-Task Learning.

Current food recommender systems tend to prioritize either the user's dietary preferences or the healthiness of the food, without considering the importance of personalized health requirements. To address this issue, we propose a novel approach to healthy food recommendations that takes into account the user's personalized health requirements, in addition to their dietary preferences. Our work comprises three perspectives. Firstly, we propose a collaborative recipe knowledge graph (CRKG) with millions of triplets, containing user-recipe interactions, recipe-ingredient associations, and other food-related information. Secondly, we define a score-based method for evaluating the healthiness match between recipes and user preferences. Based on these two prior perspectives, we develop a novel health-aware food recommendation model (FKGM) using knowledge graph embedding and multi-task learning. FKGM employs a knowledge-aware attention graph convolutional neural network to capture the semantic associations between users and recipes on the collaborative knowledge graph and learns the user's requirements in both preference and health by fusing the losses of these two learning tasks. We conducted experiments to demonstrate that FKGM outperformed four competing baseline models in integrating users' dietary preferences and personalized health requirements in food recommendations and performed best on the health task.

Knowledge graph embedding learning system for defect diagnosis in additive manufacturing

Since additive manufacturing involves complex physical processes and is influenced by many process parameters, it is challenging to manufacture defect-free products. To explore the causes of defects and obtain better product quality, exploring the relations among these factors is necessary. The knowledge graph is a powerful technique to express the relationships between entities and has been applied to many real-world tasks, such as disease diagnosis, recommendation systems, and traffic prediction. In this paper, a novel knowledge graph embedding learning system is presented, in which a designed knowledge graph ontology for AM and fact triples collected from the literature are utilized for the construction of a knowledge graph for defect diagnosis in additive manufacturing (KDDAM). A relational enhanced graph convolutional network is proposed to uncover new hidden relations among parameters, effects, defects, and product quality in the KDDAM. To validate the performance of the proposed model, we conducted many experiments on KDDAM and compared it with other baseline and state-of-the-art models. Our model yielded better results in two traditional link prediction metrics, with an MRR of 0.316 and 0.7, respectively. In addition, a case study shows that our model can discover new causes of defects to help us optimize the process.

Attributed Graph Embedding with Random Walk Regularization and Centrality-Based Attention

Graph-embedding learning is the foundation of complex information network analysis, aiming to represent nodes in a graph network as low-dimensional dense real-valued vectors for the application in practical analysis tasks. In recent years, the study of graph network representation learning has received increasing attention from researchers, and, among them, graph neural networks (GNNs) based on deep learning are playing an increasingly important role in this field. However, the fact that higher-order neighborhood information cannot be used effectively is a problem of most existing graph neural networks. Moreover, it tends to ignore the influence of latent representation and structural properties on graph embedding. In hopes of solving these issues, we introduce centrality encoding to learn the node properties, add an attention mechanism consideration to better distinguish the significance of neighboring nodes, and introduce random walk regularization to make sample neighbors that consistently satisfy predetermined criteria. This allows us to learn a representation of a potential node. We tested the performance of our model on node-clustering and link prediction tasks using three widely recognized benchmark datasets. The outcomes of our experiments demonstrate that our model significantly surpasses the baseline method in both tasks, indicating that the graph embedding it generates is highly expressive.

Fast orthogonal locality-preserving projections for unsupervised feature selection

Graph-based sparsity learning is one of the most successful unsupervised feature selection methods that has been widely adopted in many real-world applications. However, traditional graph-based unsupervised feature selection methods have several drawbacks: (1) being time-consuming and unable to deal with large-scale problems; (2) having difficulty tuning the regularization parameter with the sparsity regularization term; and (3) being unable to find explicit solutions owing to the limitation of sparsity, that is, feature selection with the ℓ2,1-norm constrained problem. Thus, this paper proposes OLPPFS, a method to preserve the local geometric structure within the feature subspace by imposing the ℓ2,0-norm constraint. First, the linear mapping capability of the proposed model is enhanced using locality-preserving projections (LPPs), whichpreserve the local and global geometric manifold structure of the data while enhancing the ability to reconstruct data. Second, the graph-embedding learning method can accelerate the construction of a sparsity affinity graph and describe the intrinsic structure of the dataset well. More importantly, we propose a method for solving a projection matrix with the ℓ2,0-norm constrained, which can accurately select a explicit group of discriminative feature subsets. This method can yield a more accurate sparse projection matrix than the ℓ2,1-norm. We also adopt FOLPPFS, an effective anchor-based strategy to further accelerate our model with two flexible options. Extensive experiments on eight datasets demonstrate that the proposed method is superior to the other methods and can preserve a better local geometric structure of the dataset with less time consumption.

PeerRemove: An adaptive node removal strategy for P2P botnet based on deep reinforcement learning

Botnets have become one of the major intrusion threats to cybersecurity. P2P botnets have high concealment and resilience because of their distributed structure, which are difficult to be completely dismantled and destroyed. Existing methods based on traffic statistics and vulnerability tracing cannot effectively solve the problem of P2P botnet disintegration. Although P2P networks emphasize that each node is peer-to-peer, the difference in processing power, resource distribution, and node bandwidth can lead to a certain heterogeneity. The critical nodes bridge the underlying bot nodes and the upper control server. Traditional methods for ranking the importance of nodes mainly relies on classical graph-theoretic feature statistics method, such as degree, betweenness, clustering coefficient, feature vector centrality, PageRank, etc. In this paper, botnet defense strategies are investigated from the perspective of complex network graph theory, and graph embedding and deep reinforcement learning combination optimization methods are adopted to handle the critical nodes identification problem of P2P botnets. Then, a novel adaptive node removal model called PeerRemove is proposed. The model uses Structure2vec graph embedding to characterize the network structure information as a low-dimensional embedding space, and it uses n-step Q-learning to train the model to learn complex topological patterns to find the critical nodes that effectively disintegrate the network. To evaluate the effectiveness of the proposed method, the Area Under the Curve (AUC) of the Largest Connected Component (LCC) size during node removal is used as an evaluation indicator, and six different types real or synthetic P2P botnets are selected, namely Sality, ZeroAccess, NSIS, Mozi, Gnutella, and Peer sampling service. Experiments are conducted on many real and model networks with node sizes reaching thousands and tens of thousands, and our method is compared with five classical static or dynamic node attack methods of HAD, PageRank, CI, BPD, and HPRA. The experimental results show that the overall AUC curve of the PeerRemove method is lower than that of the benchmark method, which can minimize botnet resiliency at a small cost. The proposed method is superior to the existing node removal methods and shows good robustness and feasibility. To demonstrate the generality of this method, it is tested on a centralized topological dataset and good experimental results are obtained.

Constructing meaningful code changes via graph transformer

AbstractThe rapid development of Open‐Source Software (OSS) has resulted in a significant demand for code changes to maintain OSS. Symptoms of poor design and implementation choices in code changes often occur, thus heavily hindering code reviewers to verify correctness and soundness of code changes. Researchers have investigated how to learn meaningful code changes to assist developers in anticipating changes that code reviewers may suggest for the submitted code. However, there are two main limitations to be addressed, including the limitation of long‐range dependencies of the source code and the missing syntactic structural information of the source code. To solve these limitations, a novel method is proposed, named Graph Transformer for learning meaningful Code Transformations (GTCT), to provide developers with preliminary and quick feedback when developers submit code changes, which can improve the quality of code changes and improve the efficiency of code review. GTCT comprises two components: code graph embedding and code transformation learning. To address the missing syntactic structural information of the source code limitation, the code graph embedding component captures the types and patterns of code changes by encoding the source code into a code graph structure from the lexical and syntactic representations of the source code. Subsequently, the code transformation learning component uses the multi‐head attention mechanism and positional encoding mechanism to address the long‐range dependencies limitation. Extensive experiments are conducted to evaluate the performance of GTCT by both quantitative and qualitative analyses. For the quantitative analysis, GTCT relatively outperforms the baseline on six datasets by 210%, 342.86%, 135%, 29.41%, 109.09%, and 91.67% in terms of perfect prediction. Meanwhile, the qualitative analysis shows that each type of code change by GTCT outperforms that of the baseline method in terms of bug fixed, refactoring code and others' taxonomy of code changes.

A Knowledge Graph Embedding Framework With Triple Semantics

The knowledge graph embedding model aims to use low-dimensional real-valued vectors to represent the entities and relations in the triples, where operations such as link prediction and triple classification can be performed based on these representations. However, existing embedding models only consider the structural embedding of triples, while ignoring the semantic information of triples. This paper proposes a knowledge graph embedding learning framework combined with triple semantic information (KGSE). KGSE comprehensively considers the structural embedding and semantic embedding of triples, where semantic embedding is used as a supplement to improve the quality of embedding. Specifically, KGSE uses the improved TransD model to obtain the structural embedding of triples, and employs the deep convolutional neural model combined with an attention mechanism to obtain the semantic embedding of triples. In addition, a novel energy function is designed to jointly train the above two embeddings. Experimental results show that the proposed framework improves significantly compared with Trans-based models and other baseline models in link prediction and triple classification tasks, which verifies the effectiveness of the proposed framework.

Assembly Sequence Optimization of Spatial Trusses Using Graph Embedding and Reinforcement Learning

We consider a truss as a graph consisting of nodes and edges, and combine graph embedding (GE) and reinforcement learning (RL) to develop an agent for generating a stable assembly path for a truss with arbitrary configuration. GE is a method of embedding the features of a graph into a vector space. By using GE, the agent can obtain numerical information on neighboring members and nodes considering their connectivity. Since the stability of a structure is strongly affected by the relative positions of members and nodes, feature extraction by GE should be effective in considering the stability of a truss. The proposed method not only can train agents using trusses with arbitrary connectivity but also can apply trained agents to trusses with arbitrary connectivity, ensuring the versatility of the trained agents' applicability. In the numerical examples, the trained agents are verified to find rational assembly sequences for various trusses more than 1000 times faster than metaheuristic approaches. The trained agent is further implemented as a user-friendly component compatible with 3D modeling software.

An Integrated Collaborative Filtering Framework With Location-Aware Graph Embedding in Intelligent Internet of Things Systems

Reliable Internet of Things (IoT) service discovery is a significant task in intelligent service-oriented IoT systems. Collaborative filtering (CF) turns out to be an effective solution to IoT service discovery. However, the traditional CF framework is facing the following challenges: inefficiency in learning the high-order interactions between users and IoT services and ineffectively making use of geographical location information. Moreover, deploying a CF framework in real-world distributed IoT systems poses another significant challenge in terms of reliability assurance and privacy protection. For bridging this gap, we propose an innovative integrated collaborative filtering framework (ICF) with incorporating the location-aware quality of IoT services into a heterogeneous graph embedding model. Meanwhile, a Geohash-based privacy-preservation mechanism is introduced to encoding the location information into a short string for protecting the sensitive location information. The proposed ICF framework is an integrated architecture combining advanced graph embedding learning and heterogeneous side information. And a joint objective optimization function is designed in the graph embedding learning to qualify automatic IoT service features. In this way, the hidden user-service interaction information and location-aware quality semantic features can be explored in an effective and reliable way. Furthermore, the learned location-aware quality embedding vector is incorporated to discover the reliable services among all IoT services with the mini-batch online clustering-based CF algorithm. The experimental findings illustrate the significant efficacy and reliability of our proposed ICF framework in large scale IoT service discovery scenarios.

A resilient network recovery framework against cascading failures with deep graph learning

Because of the increasing importance and dependencies of infrastructure networks and the potential for massive cascading failures in real-world network systems, maintenance optimization to effectively reduce system performance loss caused by diverse disruptions is of significant interest among researchers and practitioners. In this work, a new recovery framework was developed to rapidly identify important system components for maintenance to improve network resilience against cascading failures. This work provides distinct advantages to determine an optimal maintenance priority by combining real-time network structure importance with other maintenance prioritization based on customer preference. This approach adopts structural graph embedding and deep reinforcement learning to extract real-time network topology information (such as minimum vertex cover) to update the maintenance priority during the recovery process. Based on the case studies on synthetic networks and a US airport network, the proposed recovery framework with real-time network topology awareness shows better performance than other maintenance prioritization strategies regarding resilience enhancement. This work improves the understanding of how the changing network structure influences maintenance effects. It also provides insights of the practical usefulness of advanced deep learning on helping optimal maintenance prioritization to effectively reduce the intensity and extent of cascading failures.

Urban Ecological Monitoring and Prediction Based on Deep Learning

Obtaining comprehensive and accurate air quality information is conducive to people’s daily travel and living arrangements, especially to protect people’s health from air pollutants. Due to the limited number of air quality monitoring stations and the lack of training samples, the generalisation performance of air quality estimation model is often not good enough. Therefore, we propose an urban air quality index (AQI) prediction and AQI level estimation method based on deep multi-task learning. We consider various urban big data information related to air quality (meteorology, transportation, enterprise self-test, POI, road network, etc.), and use machine learning methods such as deep learning and graph embedding learning to learn the representation of relevant information, and establish the relationship between these related representations and air quality. Experiments show that this scheme can estimate the level of urban air quality index joint prediction task and air quality index, and the model has generalisation performance.