Graph Learning Research Articles

Human learning of hierarchical graphs.

Humans are exposed to sequences of events in the environment, and the interevent transition probabilities in these sequences can be modeled as a graph or network. Many real-world networks are organized hierarchically and while much is known about how humans learn basic transition graph topology, whether and to what degree humans can learn hierarchical structures in such graphs remains unknown. We probe the mental estimates of transition probabilities via the surprisal effect phenomenon: humans react more slowly to less expected transitions. Using mean-field predictions and numerical simulations, we show that surprisal effects are stronger for finer-level than coarser-level hierarchical transitions, and that surprisal effects at coarser levels are difficult to detect for limited learning times or in small samples. Using a serial response experiment with human participants (n=100), we replicate our predictions by detecting a surprisal effect at the finer level of the hierarchy but not at the coarser level of the hierarchy. We then evaluate the presence of a trade-off in learning, whereby humans who learned the finer level of the hierarchy better also tended to learn the coarser level worse, and vice versa. This study elucidates the processes by which humans learn sequential events in hierarchical contexts. More broadly, our work charts a road map for future investigation of the neural underpinnings and behavioral manifestations of graph learning.

Spatiotemporal gated traffic trajectory simulation with semantic-aware graph learning

Traffic trajectories of various vehicles, bicycles and pedestrians can help understand the traffic dynamics in a fine-grained manner like traffic flow, traffic congestion and ride-hailing demand. The comprehensive usage of traffic trajectory data has not been fully investigated due to the prevalent privacy concerns and commercial limitations. The traffic trajectory simulation task has emerged to generate high-fidelity trajectories in demand for downstream tasks to fill the gap between the scarce trajectory data and the widespread applications. Previous state-of-the-art methods build the spatiotemporal dependencies of trajectories with Graph Neural Networks (GNNs) under generative adversarial training, yielding better yet unstable trajectory quality. We observe that the unsatisfied synthetic trajectories are caused by the insufficient spatiotemporal modeling of road networks and trajectory semantics. In this paper, we propose a novel SpatioTEmporal GAted (STEGA) framework equipped with semantic-aware graph learning for traffic trajectory simulation to enable the explicit modeling of spatiotemporal dependencies throughout the learning pipeline. On the one hand, STEGA employs a graph encoder with the semantics of road networks for the spatial points of a trajectory, together with a time encoder for the time points. On the other hand, STEGA devises two spatiotemporal gates with the semantic graphs for the predictions of the future trajectory. Boosted by the semantic-aware graph learning, the proposed STEGA outperforms the state-of-the-art counterparts consistently at both macro- and micro-level metrics on two datasets. Elaborate ablation studies and downstream tasks of the synthetic trajectories further demonstrate the superiority of STEGA. Our code is available at https://github.com/Star607/STEGA.

Deep dynamic high-order graph convolutional network for wear fault diagnosis of hydrodynamic mechanical seal

The hydrodynamic mechanical seal (HDMS) in the reactor coolant pump of third-generation nuclear power units is vulnerable to failure due to prolonged operational periods and inevitable wear. However, traditional fault diagnosis methods are not robust to noise and can not leverage both the topological relationships among samples and local features. To resolve these challenges, in this paper, we propose a novel graph convolutional network (GCN) for wear fault diagnosis of HDMS called deep dynamic high-order graph convolutional network (DDHGCN). A dynamic graph learning module is designed to control the connectivity and sparsity of the iterated graph and thus eliminate errors and redundancies caused by noise. A high-order GCN module is proposed to effectively model the correlations between nodes, capturing contextual information and mutual influences among them. A residual convolutional module is applied to extract local features hidden in individual samples to further improve the classification performance. All three modules are jointly optimized for reliable wear fault diagnosis of HDMS. Experimental results demonstrate that our DDHGCN can achieve higher performance when compared with the state-of-the-arts.

SAGL: A self-attention-based graph learning framework for predicting survival of colorectal cancer patients

Background and Objective: Colorectal cancer (CRC) is one of the most commonly diagnosed cancers worldwide. The accurate survival prediction for CRC patients plays a significant role in the formulation of treatment strategies. Recently, machine learning and deep learning approaches have been increasingly applied in cancer survival prediction. However, most existing methods inadequately represent and leverage the dependencies among features and fail to sufficiently mine and utilize the comorbidity patterns of CRC. To address these issues, we propose a self-attention-based graph learning (SAGL) framework to improve the postoperative cancer-specific survival prediction for CRC patients.Methods: We present a novel method for constructing dependency graph (DG) to reflect two types of dependencies including comorbidity-comorbidity dependencies and the dependencies between features related to patient characteristics and cancer treatments. This graph is subsequently refined by a disease comorbidity network, which offers a holistic view of comorbidity patterns of CRC. A DG-guided self-attention mechanism is proposed to unearth novel dependencies beyond what DG offers, thus augmenting CRC survival prediction. Finally, each patient will be represented, and these representations will be used for survival prediction.Results: The experimental results show that SAGL outperforms state-of-the-art methods on a real-world dataset, with the receiver operating characteristic curve for 3- and 5-year survival prediction achieving 0.849±0.002 and 0.895±0.005, respectively. In addition, the comparison results with different graph neural network-based variants demonstrate the advantages of our DG-guided self-attention graph learning framework.Conclusions: Our study reveals that the potential of the DG-guided self-attention in optimizing feature graph learning which can improve the performance of CRC survival prediction.

Adaptive orthogonal semi-supervised feature selection with reliable label matrix learning

Feature selection (FS) can select features with high value from high-dimensional data as much as possible and reduce the dimensionality of the data to improve the performance of the machine learning model and enhance the generalization ability. In semi-supervised FS (SSFS), regression-based methods utilize the label matrix, and the quality of the label matrix can directly affect the performance of FS. Hence, the importance of the reliable label matrix is obvious. Orthogonal regression (OR) can retain more information in the subspace than least squares regression (LSR). Therefore, this paper introduces OR into SSFS, and then the label scaling technique is used to learn a reliable label matrix. Also, this study utilizes adaptive graph learning to exploit more structural information about the data. Two constraints, Frobenius-norm or maximum information entropy imposed on the similarity matrix and two adaptive orthogonal SSFS (AGLSOFS) methods with reliable label matrix learning are constructed. The impact of these two constraints on the construction of dynamic similarity graphs and FS results is discussed. Effective optimization algorithms for these two methods are based on the Alternating Direction Method of Multipliers (ADMM) and Generalized Power Iteration (GPI). Experiments are conducted on 15 benchmark datasets, and the results show that: (1) similarity graphs constructed using both original and projected data are more accurate; (2) both constraints are valid; (3) both of the methods of this paper perform well on most datasets; (4) OR performs better than LSR for FS; and (5) the scaling factor affects the convergence speed of the model.

Attributed Graph Force Learning.

In numerous network analysis tasks, feature representation plays an imperative role. Due to the intrinsic nature of networks being discrete, enormous challenges are imposed on their effective usage. There has been a significant amount of attention on network feature learning in recent times that has the potential of mapping discrete features into a continuous feature space. The methods, however, lack preserving the structural information owing to the utilization of random negative sampling during the training phase. The ability to effectively join attribute information to embedding feature space is also compromised. To address the shortcomings identified, a novel attribute force-based graph (AGForce) learning model is proposed that keeps the structural information intact along with adaptively joining attribute information to the node's features. To demonstrate the effectiveness of the proposed framework, comprehensive experiments on benchmark datasets are performed. AGForce based on the spring-electrical model extends opportunities to simulate node interaction for graph learning.

Spectral Graph Convolutional Neural Networks in the Context of Regularization Theory.

Graph convolutional neural networks (GCNN) have been widely used in graph learning and related applications. It has been identified that the filters in the state-of-the-art spectral graph convolutional networks (SGCNs) are essentially low-pass filters that enforce smoothness across the graph and use the functions of graph Laplacian as a tool that injects graph structure into the learning algorithm. There had been research findings that connect the smoothness functional in graphs, graph Laplacian, and regularization operators. We review the existing SGCNs in this context and propose a framework where the state-of-the-art filter designs can be deduced as special cases. We designed new filters that are associated with well-defined low-pass behavior and tested their performance on semisupervised node classification tasks. Their performance was found to be superior to that of the other state-of-the-art techniques. We further investigate the representation capability of low-pass features and make useful observations. In this context, we discuss a few points to further optimize the network, new strategies for designing SGCNs, their challenges, and some latest related developments. Based on our framework, we also deduce the connection between support vector kernels and SGCN filters.

Enforced Block Diagonal Graph Learning for Multikernel Clustering

The existing multikernel graph clustering (MKGC) methods have emerged with notable success on nonlinear clustering tasks since the graph learning can effectively capture graph structure similarity between sample pair. Ideally, a high-quality graph should enjoy the good block diagonal property, i.e. the intercluster similarities correspond to zeros, while intracluster similarities represent nonzeros. Meanwhile, the number of diagonal blocks for a graph equals to the number of clusters on the dataset. However, most of the existing MKGC methods design a corpulent three-parts graph leaning process that poses challenges for hyperparameter tuning, time cost, and clustering performance. To overcome these challenging issues, we propose an enforced block diagonal graph learning for multikernel clustering (EBDGL-MKC) method, where we pursue a high-quality block diagonal graph via well-designed one-part graph leaning scheme rather than three parts. Inspired by symmetric matrix factorization (SMF), we first design a one-part block diagonal graph learning scheme to learn multiple block diagonal graphs, by exploring an explicit theoretical connection between the clustering partition of kernel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> -means and the excellent block diagonal graph. Then, these block diagonal graphs are stacked into a low-rank tensor for exploiting the high-order structure information hidden in the nonlinear data. After that, an effective alternate algorithm with convergence proof is performed on extensive experiments to demonstrate the superiority of EBDGL compared with the state-of-the-art multikernel clustering (MKC) methods.

Reverse Graph Learning for Graph Neural Network.

Graph neural networks (GNNs) conduct feature learning by taking into account the local structure preservation of the data to produce discriminative features, but need to address the following issues, i.e., 1) the initial graph containing faulty and missing edges often affect feature learning and 2) most GNN methods suffer from the issue of out-of-example since their training processes do not directly generate a prediction model to predict unseen data points. In this work, we propose a reverse GNN model to learn the graph from the intrinsic space of the original data points as well as to investigate a new out-of-sample extension method. As a result, the proposed method can output a high-quality graph to improve the quality of feature learning, while the new method of out-of-sample extension makes our reverse GNN method available for conducting supervised learning and semi-supervised learning. Experimental results on real-world datasets show that our method outputs competitive classification performance, compared to state-of-the-art methods, in terms of semi-supervised node classification, out-of-sample extension, random edge attack, link prediction, and image retrieval.

SGT++: Improved Scene Graph-Guided Transformer for Surgical Report Generation.

Automatically recording surgical procedures and generating surgical reports are crucial for alleviating surgeons' workload and enabling them to concentrate more on the operations. Despite some achievements, there still exist several issues for the previous works: 1) failure to model the interactive relationship between surgical instruments and tissue; and 2) neglect of fine-grained differences within different surgical images in the same surgery. To address these two issues, we propose an improved scene graph-guided Transformer, also named by SGT++, to generate more accurate surgical report, in which the complex interactions between surgical instruments and tissue are learnt from both explicit and implicit perspectives. Specifically, to facilitate the understanding of the surgical scene graph under a graph learning framework, a simple yet effective approach is proposed for homogenizing the input heterogeneous scene graph. For the homogeneous scene graph that contains explicit structured and fine-grained semantic relationships, we design an attention-induced graph transformer for node aggregation via an explicit relation-aware encoder. In addition, to characterize the implicit relationships about the instrument, tissue, and the interaction between them, the implicit relational attention is proposed to take full advantage of the prior knowledge from the interactional prototype memory. With the learnt explicit and implicit relation-aware representations, they are then coalesced to obtain the fused relation-aware representations contributing to generating reports. Some comprehensive experiments on two surgical datasets show that the proposed STG++ model achieves state-of-the-art results.

Self-Completed Bipartite Graph Learning for Fast Incomplete Multi-View Clustering

Self-Completed Bipartite Graph Learning for Fast Incomplete Multi-View Clustering

Dynamic Relation Graph Learning for Time-Aware Service Recommendation

Dynamic Relation Graph Learning for Time-Aware Service Recommendation

Multi-Flow Transmission in Wireless Interference Networks: A Convergent Graph Learning Approach

Multi-Flow Transmission in Wireless Interference Networks: A Convergent Graph Learning Approach

Imbalanced Large Graph Learning Framework for FPGA Logic Elements Packing Prediction

Imbalanced Large Graph Learning Framework for FPGA Logic Elements Packing Prediction

Continuous Training of Recommendation System for Airbnb Listings Using Graph Learning

Recommender systems are getting increasingly important nowadays as they can boost user engagement and benefit businesses. However, there remain some unsolved problems. This paper will address two key performance issues. First, the limited ability to identify and leverage intrinsic relationships between data points. Second, the inability to adapt to new data. The first issue is proposed to be addressed through a Graph Neural Network (GNN) to curate better recommendations. GNN will be trained with Airbnb’s review data to utilize its outstanding expressive power to represent complex user-listing interactions at scale, followed by generating embeddings to compute the relevant recommendations to the users. With the generated embeddings, the recommender system will compute a recommendation list to every user based on the embedding similarity between the user and listings or the user’s first-ever reviewed listing and listings. The second issue is proposed to be resolved by incorporating Continuous Training. The proposed recommender system employs GraphSAGE with a customized Rating-Weighted Triplet Ranking Loss function, which outperformed unsupervised GraphSAGE. Offline simulation validated the recommender system's ability to learn from the latest data and improve over time. Overall, the proposed user-to-item (U2I) recommendation rating-weighted GraphSAGE substantially increased by 99.88% in hit-rate@5 and 98.15% in coverage. This offers an effective solution for enhancing the recommender system for Airbnb listings. This research validates the efficacy of GNN-based recommendations in capturing user-item relationships to aid in predicting relevant recommendations, thus significantly driving up the adoption of GNN-based recommender systems.

Measuring drug similarity using drug–drug interactions

AbstractCombination therapy is a promising approach to address the challenge of antimicrobial resistance, and computational models have been proposed for predicting drug–drug interactions. Most existing models rely on drug similarity measures based on characteristics such as chemical structure and the mechanism of action. In this study, we focus on the network structure itself and propose a drug similarity measure based on drug–drug interaction networks. We explore the potential applications of this measure by combining it with unsupervised learning and semi‐supervised learning approaches. In unsupervised learning, drugs can be grouped based on their interactions, leading to almost monochromatic group–group interactions. In addition, drugs within the same group tend to have similar mechanisms of action (MoA). In semi‐supervised learning, the similarity measure can be utilized to construct affinity matrices, enabling the prediction of unknown drug–drug interactions. Our method exceeds existing approaches in terms of performance. Overall, our experiments demonstrate the effectiveness and practicability of the proposed similarity measure. On the one hand, when combined with clustering algorithms, it can be used for functional annotation of compounds with unknown MoA. On the other hand, when combined with semi‐supervised graph learning, it enables the prediction of unknown drug–drug interactions.

Graph neural network-based anomaly detection for river network systems

Background Water is the lifeblood of river networks, and its quality plays a crucial role in sustaining both aquatic ecosystems and human societies. Real-time monitoring of water quality is increasingly reliant on in-situ sensor technology. Anomaly detection is crucial for identifying erroneous patterns in sensor data, but can be a challenging task due to the complexity and variability of the data, even under typical conditions. This paper presents a solution to the challenging task of anomaly detection for river network sensor data, which is essential for accurate and continuous monitoring. Methods We use a graph neural network model, the recently proposed Graph Deviation Network (GDN), which employs graph attention-based forecasting to capture the complex spatio-temporal relationships between sensors. We propose an alternate anomaly threshold criteria for the model, GDN+, based on the learned graph. To evaluate the model’s efficacy, we introduce new benchmarking simulation experiments with highly-sophisticated dependency structures and subsequence anomalies of various types. We also introduce software called gnnad. Results We further examine the strengths and weaknesses of this baseline approach, GDN, in comparison to other benchmarking methods on complex real-world river network data. Conclusions Findings suggest that GDN+ outperforms the baseline approach in high-dimensional data, while also providing improved interpretability.

Research on Blockchain Transaction Privacy Protection Methods Based on Deep Learning

To address the challenge of balancing privacy protection with regulatory oversight in blockchain transactions, we propose a regulatable privacy protection scheme for blockchain transactions. Our scheme utilizes probabilistic public-key encryption to obscure the true identities of blockchain transaction participants. By integrating commitment schemes and zero-knowledge proof techniques with deep learning graph neural network technology, it provides privacy protection and regulatory analysis of blockchain transaction data. This approach not only prevents the leakage of sensitive transaction information, but also achieves regulatory capabilities at both macro and micro levels, ensuring the verification of the legality of transactions. By adopting an identity-based encryption system, regulatory bodies can conduct personalized supervision of blockchain transactions without storing users’ actual identities and key data, significantly reducing storage computation and key management burdens. Our scheme is independent of any particular consensus mechanism and can be applied to current blockchain technologies. Simulation experiments and complexity analysis demonstrate the practicality of the scheme.

Product Space Clustering with Graph Learning for Diversifying Industrial Production

During economic crises, diversifying industrial production emerges as a critical strategy to address societal challenges. The Product Space, a graph representing industrial knowledge proximity, acts as a valuable tool for recommending diversified product offerings. These recommendations rely on the edges of the graph to identify suitable products. They can be improved by grouping similar products together, which results in more precise suggestions. Unlike the topology, the textual data in nodes of the Product Space graph are typically unutilized in graph clustering methods. In this context, we propose a novel approach for economic graph learning that incorporates learning node data alongside network topology. By applying this method to the Product Space dataset, we demonstrate how recommendations have been improved by presenting real-life applications. Our research employing a graph neural network demonstrates superior performance compared to methods like Louvain and I-Louvain. Our contribution introduces a node data-based deep graph clustering graph neural network that significantly advances the macroeconomic literature and addresses the imperative of diversifying industrial production. We discuss both the advantages and limitations of deep graph learning models in economics, laying the groundwork for future research.

A physics-informed graph learning approach for citywide electric vehicle charging demand prediction and pricing

A growing number of electric vehicles (EVs) is putting pressure on smart charging services. As a foundation of informing drivers of vacant charging facilities and rationalizing pricing, an effective approach for predicting citywide spatio-temporal EV charging demand that incorporates pricing information is required. Although many deep learning models have been carried out with the orientation of improving prediction accuracy and achieved good results, there is still a lack of prediction models that are economically interpretable and can consider the spillover effect of price adjustments on EV charging demand. To fill the research gaps, we propose a learning approach for accurate EV charging demand prediction and reasonable pricing, named PIAST, which enables the integration of convolutional feature engineering, spatio-temporal dual attention mechanism and physics-informed neural network training. On a dataset containing 18,061 EV charging piles, we demonstrate the state-of-the-art performance of the proposed model. The results empirically showed that the proposed approach achieved an average improvement of 14.78% in accuracy, compared to other representative models. Moreover, the model can uncover the price elasticity of demand during training, making the model interpretable. Finally, price adjustment experiments are conducted to demonstrate the self-influence and spillover effects of price.