Inductive Graph Research Articles

Link Stealing Attacks Against Inductive Graph Neural Networks

A graph neural network (GNN) is a type of neural network that is specifically designed to process graph-structured data. Typically, GNNs can be implemented in two settings, including the transductive setting and the inductive setting. In the transductive setting, the trained model can only predict the labels of nodes that were observed at the training time. In the inductive setting, the trained model can be generalized to new nodes/graphs. Due to its flexibility, the inductive setting is the most popular GNN setting at the moment. Previous work has shown that transductive GNNs are vulnerable to a series of privacy attacks. However, a comprehensive privacy analysis of inductive GNN models is still missing. This paper fills the gap by conducting a systematic privacy analysis of inductive GNNs through the lens of link stealing attacks. We propose two types of link stealing attacks, i.e., posterior-only attacks and combined attacks. We define threat models of the posterior-only attacks with respect to node topology and the combined attacks by considering combinations of posteriors, node attributes, and graph features. Extensive evaluation on six real-world datasets demonstrates that inductive GNNs leak rich information that enables link stealing attacks with advantageous properties. Even attacks with no knowledge about graph structures can be effective. We also show that our attacks are robust to different node similarities and different graph features. As a counterpart, we investigate two possible defenses and discover they are ineffective against our attacks, which calls for more effective defenses.

A framework armed with node dynamics for predicting technology convergence

In the rapidly evolving landscape of industrial and societal progress, technology convergence plays a pivotal role. This dynamic process is usually characterized by the emergence of new nodes and new links. With the long-term and recent interests in predicting technology convergence, link prediction has become the primary approach on the basis of large-scale patent data. Though, the problem of node dynamics is still not addressed in the literature. For this purpose, this paper presents a technology convergence prediction framework with three core modules as follows. (1) A candidate node set is introduced during the network construction phase, mimicking the generation of newly-emerging nodes. (2) An inductive graph representation learning approach is deployed to generate feature vectors for newly-emerging nodes as well as existing ones. (3) The evaluation criteria are revised to shift from the predictable range to the actual predicted range, which can provide a more realistic assessment of predictive performance. Finally, experimental results on the domain of cancer drug development validate the feasibility and effectiveness of our framework in capturing the dynamics of technology convergence, especially concerning the relationships of newly emerged nodes and links. This study provides valuable insights into technology convergence dynamics and points to future research and applications.

SHINE: A Scalable Heterogeneous Inductive Graph Neural Network for Large Imbalanced Datasets

SHINE: A Scalable Heterogeneous Inductive Graph Neural Network for Large Imbalanced Datasets

TIGER: Training Inductive Graph Neural Network for Large-Scale Knowledge Graph Reasoning

Knowledge Graph (KG) Reasoning plays a vital role in various applications by predicting missing facts from existing knowledge. Inductive KG reasoning approaches based on Graph Neural Networks (GNNs) have shown impressive performance, particularly when reasoning with unseen entities and dynamic KGs. However, such state-of-the-art KG reasoning approaches encounter efficiency and scalability challenges on large-scale KGs due to the high computational costs associated with subgraph extraction - a key component in inductive KG reasoning. To address the computational challenge, we introduce TIGER, an inductive GNN training framework tailored for large-scale KG reasoning. TIGER employs a novel, efficient streaming procedure that facilitates rapid subgraph slicing and dynamic subgraph caching to minimize the cost of subgraph extraction. The fundamental challenge in TIGER lies in the optimal subgraph slicing problem, which we prove to be NP-hard. We propose a novel two-stage algorithm SiGMa to solve the problem practically. By decoupling the complicated problem into two classical ones, SiGMa achieves low computational complexity and high slice reuse. We also propose four new benchmarks for robust evaluation of large-scale inductive KG reasoning, the biggest of which performs on the Freebase KG (encompassing 86M entities, 285M edges). Through comprehensive experiments on state-of-the-art GNN-based KG reasoning models, we demonstrate that TIGER significantly reduces the running time of subgraph extraction, achieving an average 3.7× speedup relative to the basic training procedure.

Hybrid Inductive Graph Method for Matrix Completion

The recommender system can be viewed as a matrix completion problem, which aims to predict unknown values within a matrix. Solutions to this problem are categorized into two approaches: transductive and inductive reasoning. In transductive reasoning, the model cannot be applied to new cases unseen during training. In contrast, IGMC, the state-of-the-art inductive algorithm, only requires subgraphs for target users and items, without needing any other content information. While the absence of a requirement for content information simplifies the model and enhances transferability to new tasks, incorporating content information could still improve the model's performance. In this article, the authors introduce Hi-GMC, a hybrid version of the IGMC model that incorporates content information alongside users and items. They present a novel graph model to encapsulate the side information related to users and items and develop a learning method based on graph neural networks. This proposed method achieves state-of-the-art performance on the MovieLens-100K dataset for both warm and cold start scenarios.

Large-scale graph-machine-learning surrogate models for 3D-flowfield prediction in external aerodynamics

The article presents the application of inductive graph machine learning surrogate models for accurate and efficient prediction of 3D flow for industrial geometries, explicitly focusing here on external aerodynamics for a motorsport case. The final aim is to build a surrogate model that can provide quick predictions, bypassing in this way the unfeasible computational burden of traditional computational fluid dynamics (CFD) simulations. We investigate in this contribution the usage of graph neural networks, given their ability to smoothly deal with unstructured data, which is the typical context for industrial simulations. We integrate an efficient subgraph-sampling approach with our model, specifically tailored for large dataset training. REV-GNN is the chosen graph machine learning model, that stands out for its capacity to extract deeper insights from neighboring graph regions. Additionally, its unique feature lies in its reversible architecture, which allows keeping the memory usage constant while increasing the number of network layers. We tested the methodology by applying it to a parametric Navier–Stokes problem, where the parameters control the surface shape of the industrial artifact at hand, here a motorbike.

JORA: Weakly Supervised User Identity Linkage via Jointly Learning to Represent and Align.

The user identity linkage that establishes correspondence between users across networks is a fundamental issue in various social network applications. Efforts have recently been devoted to introducing network embedding techniques that map the different network users into the common representation space, thereby inferring user correspondence based on the similarities of their representations. However, existing studies that separately train the network embedding and space alignment in two stages may lead to conflict between the objectives of the two stages. Besides, the similarities between unlabeled cross-network user pairs are difficult to define and largely impact the result. Moreover, many previous methods still need plenty of labeled aligned user pairs to ensure performance, which may not be available. To address the above problems, we propose to solve the weakly-supervised user identity linkage problem via JOintly learning to Represent and Align, i.e., the JORA model. The architecture of JORA adopts the inductive graph convolutional network (GCN) that learns representations for each network. The model is jointly optimized by the representation learning component and alignment learning component. The former one aims to preserve the similarities between intranetwork users. The latter one aligns the different spaces by a projection function and aims to preserve the similarities between cross-network users. A specific attention mechanism is proposed to learn self-adaptive similarities for unlabeled user pairs during alignment learning and it reduces the error propagation caused by predefined similarities. The joint optimization helps perceive network characteristics during alignment and reduces the number of labeled users required. Experiments conducted on real social networks show that the proposed model achieves significantly better performance than the state-of-the-art methods.

Inductive and adaptive graph convolution networks equipped with constraint task for spatial–temporal traffic data kriging

In intelligent transportation systems (ITS), deploying fine-grained sensors to continuously collect spatial–temporal traffic data is important but impractical due to the expensive cost. Fortunately, spatial–temporal kriging methods bring advanced solutions for interpolating traffic data for locations without sensors, but they still have the following two drawbacks: (1) The widely adopted predefined and adaptive graphs are either inflexible or limited to transductive learning. (2) The sampling strategies to support inductive learning on new graphs result in losing important partial information. To overcome the above issues, in this paper, we propose an Inductive and Adaptive Graph Convolution Networks (IAGCN) for spatial–temporal traffic data kriging in an inductive manner. Specifically, we propose an adaptive graph constructor to model the hidden spatial relation of nodes and learn the spatial dependency that the predefined graph structure cannot capture. It can work inductively and does not require retraining when introducing new nodes. Additionally, we design a framework that integrates inductive graph convolution and temporal convolution to simultaneously capture the complex spatial–temporal dependencies of traffic data. Finally, to address the information loss issue caused by random sampling, we design a predicted-based constraint task that perceives and utilizes the history information of all observed sensors to predict the current data, as well as approach the result of the interpolation and prediction. Experiments on four real-world datasets show that IAGCN outperforms the state-of-the-art baselines.

Keyword-enhanced recommender system based on inductive graph matrix completion

Going beyond the user–item rating information, recent studies have utilized additional information to improve the performance of recommender systems. Graph neural network (GNN) based approaches are among the most common. However, existing models that utilize text data require a lot of computing resources and have a complex structure that makes them difficult to utilize in real-world applications. In this research, we propose a new method, keyword-enhanced graph matrix completion (KGMC), which utilizes keyword sharing relationships in user–item graphs. Our model has a simpler structure and requires less computing resources than existing models that utilize text data, but it has the advantage of cross-domain transferability while providing an intuitive understanding of the inference results. KGMC consists of three steps: (1) keyword extraction from the review text, (2) subgraph extraction and keyword-enhanced subgraph construction, and (3) GNN-based rating prediction. We have conducted extensive experiments over eight benchmark datasets to examine the relative superiority of the proposed KGMC method, compared to state-of-the-art baselines. Additional experiments and case studies have been also conducted to demonstrate the transferability as well as keyword-based explainability of KGMC. Our findings highlight the practical advantages of our model for recommender systems and support its effectiveness in inductive graph-based link prediction.

Temporal enhanced inductive graph knowledge tracing

Temporal enhanced inductive graph knowledge tracing

Syntax-based dynamic latent graph for event relation extraction

This paper focuses on extracting temporal and parent–child relationships between news events in social news. Previous methods have proved that syntactic features are valid. However, most previous methods directly use the static outcomes parsed by syntactic parsing tools, but task-irrelevant or erroneous parses will inevitably degrade the performance of the model. In addition, many implicit higher-order connections that are directly related and critical to tasks are not explicitly exploited. In this paper, we propose a novel syntax-based dynamic latent graph model (SDLG) for this task. Specifically, we first apply a syntactic type-enhanced attention mechanism to assign different weights to different connections in the parsing results, which helps to filter out noisy connections and better fuse the information in the syntactic structures. Next, we introduce a dynamic event pair-aware induction graph to mine the task-related latent connections. It constructs a potential attention matrix to complement and correct the supervised syntactic features, using the semantics of the event pairs as a guide. Finally, the latent graph, together with the syntactic information, is fed into the graph convolutional network to obtain an improved representation of the event to complete relational reasoning. We have conducted extensive experiments on four public benchmarks, MATRES, TCR, HiEve and TB-Dense. The results show that our model outperforms the state-of-the-art model by 0.4%, 1.5%, 3.0% and 1.3% in F1 scores on the four datasets, respectively. Finally, we provide detailed analyses to show the effectiveness of each proposed component.

Spatial-Temporal Aware Inductive Graph Neural Network for C-ITS Data Recovery

With the prevalence of Intelligent Transportation Systems (ITS), massive sensors are deployed on roadside, vehicles, and infrastructures. One key challenge is imputing several different types of missing entries in spatial-temporal traffic data to meet the high-quality demand of data science applied in Cooperative-ITS (C-ITS) since accurate data recovery is critical to many downstream tasks in ITSs, such as traffic monitoring and decision making. For such, it is proposed in this article solutions to three kinds of data recovery tasks in a unified model via spatial-temporal aware Graph Neural Networks (GNNs), named Spatial-Temporal Aware Data Recovery Network (STAR), enabling a real-time and inductive inference. A residual gated temporal convolution network is designed to permit the proposed model to learn the temporal pattern from long sequences with masks and an adaptive memory-based attention model for utilizing implicit spatial correlation. To further exploit the generalization power of GNNs, a sampling-based method is adopted to train the proposed model to be robust and inductive for online servicing. Extensive numerical experiments on two real-world spatial-temporal traffic datasets are performed, and results show that the proposed STAR model consistently outperforms other baselines at 1.5-2.5 times on all kinds of imputation tasks. Moreover, STAR can support recovery data for 2 to 5 hours, with its performance barely unchanged, and has comparable performance in transfer learning and time-series forecast. Experimental results demonstrate that STAR provides adequate performance and rich features for multiple data recovery tasks under the C-ITS scenario.

PO-GNN: Position-observant inductive graph neural networks for position-based prediction

Most graph neural networks (GNNs) map local neighborhood information into node embeddings but lack positional information. It results in similar node embeddings for nodes in similar local neighborhoods, leading to poor performance. We propose a position-observant inductive graph neural network (PO-GNN) architecture that captures the global positions of the nodes using the proposed novel closeness-distance metric. We demonstrate that the problem of anchor node selection, a significant component of position-aware GNNs, is NP-hard. Additionally, we utilize a heuristic anchor selection approach to obtain equitably distributed and asymmetric reference nodes over the given graph. In particular, PO-GNN significantly achieved a 50% relative improvement in the AUC metric for the Link Prediction (LP) over five different datasets and 85% for the Pairwise Node Classification (PNC) tasks over four different datasets and compared to the common GNN architectures and 1.805% and 15.29% improvement over state-of-the-art position-aware GNNs in LP and PNC tasks respectively. Furthermore, we also perform ablation studies over the length and number of random walks along with five aggregation methods and four update methods.

SpecKriging: GNN-Based Secure Cooperative Spectrum Sensing

Cooperative spectrum sensing (CSS) adopted by spectrum-sensing providers (SSPs) plays a key role for dynamic spectrum access and is essential for avoiding interference with licensed primary users (PUs). A typical SSP system consists of geographically distributed spectrum sensors which can be compromised to submit fake spectrum-sensing reports. In this paper, we propose SpecKriging, a new spatial-interpolation technique based on Inductive Graph Neural Network Kriging (IGNNK) for secure CSS. In SpecKriging, we first pretrain a graphical neural network (GNN) model with the historical sensing records of a few trusted anchor sensors. During system runtime, we use the trained model to evaluate the trustworthiness of non-anchor sensors’ data and also use them along with anchor sensors’ new data to retrain the model. SpecKriging outputs trustworthy sensor reports for spectrum-occupancy detection. To the best of our knowledge, SpecKriging is the first work that explores GNNs for trustworthy CSS and also incorporates the hardware heterogeneity of spectrum sensors. Extensive experiments confirm the high efficacy and efficiency of SpecKriging for trustworthy spectrum-occupancy detection even when malicious spectrum sensors constitute the majority.

Partial Label Learning Based on Disambiguation Correction Net With Graph Representation

Partial Label Learning (PLL) is a weakly supervised learning framework where each training instance is associated with more than one candidate label. This learning method is dedicated to finding out the true label for each training instance. Most of the current PLL algorithms directly disambiguate the candidate labels without correcting the disambiguated results, making the algorithms vulnerable to the influence of instances easily misjudged. In this paper, GraphDCN, an innovative disambiguation correction net with inductive graph representation learning model is proposed. GraphDCN consists of a disambiguation model and a correction model. For a given instance, the disambiguation model tries to fit its underlying ground-truth label through the candidate label distribution of the instances connected with the given one, while the correction model tries to maximize the distance between the disambiguated labels and non-candidate labels, and uses the label probability thresholds to correct the disambiguated labels that may be wrong. As the training goes on, both the disambiguation and correction models alternately and iteratively boost their performance. Moreover, when considering the implementation of the disambiguation model, a partial cross entropy formulation is proposed to estimate the ground-truth label loss by updating the ambiguity confidence matrix, which can be proved satisfying convergence in PLL. Simulation results reveal the overwhelming performance of GraphDCN.

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

Neural architecture search (NAS) has seen significant attention throughout the computational intelligence research community and has pushed forward the state-of-the-art of many neural models to address grid-like data such as texts and images. However, little work has been done on Graph Neural Network (GNN) models dedicated to unstructured network data. Given the huge number of choices and combinations of components such as aggregators and activation functions, determining the suitable GNN model for a specific problem normally necessitates tremendous expert knowledge and laborious trials. In addition, the moderate change of hyperparameters such as the learning rate and dropout rate would dramatically impact the learning capacity of a GNN model. In this paper, we propose a novel framework through the evolution of individual models in a large GNN architecture searching space. Instead of simply optimizing the model structures, an alternating evolution process is performed between GNN model structures and hyperparameters to dynamically approach the optimal fit of each other. Experiments and validations demonstrate that evolutionary NAS is capable of matching existing state-of-the-art reinforcement learning methods for both transductive and inductive graph representation learning and node classification.

An inductive graph neural network model for compound-protein interaction prediction based on a homogeneous graph.

Identifying the potential compound–protein interactions (CPIs) plays an essential role in drug development. The computational approaches for CPI prediction can reduce time and costs of experimental methods and have benefited from the continuously improved graph representation learning. However, most of the network-based methods use heterogeneous graphs, which is challenging due to their complex structures and heterogeneous attributes. Therefore, in this work, we transformed the compound–protein heterogeneous graph to a homogeneous graph by integrating the ligand-based protein representations and overall similarity associations. We then proposed an Inductive Graph AggrEgator-based framework, named CPI-IGAE, for CPI prediction. CPI-IGAE learns the low-dimensional representations of compounds and proteins from the homogeneous graph in an end-to-end manner. The results show that CPI-IGAE performs better than some state-of-the-art methods. Further ablation study and visualization of embeddings reveal the advantages of the model architecture and its role in feature extraction, and some of the top ranked CPIs by CPI-IGAE have been validated by a review of recent literature. The data and source codes are available at https://github.com/wanxiaozhe/CPI-IGAE.

Inductive Representation Learning on Dynamic Stock Co-Movement Graphs for Stock Predictions

Co-movement among individual firms’ stock prices can reflect complex interfirm relationships. This paper proposes a novel method to leverage such relationships for stock price predictions by adopting inductive graph representation learning on dynamic stock graphs constructed based on historical stock price co-movement. To learn node representations from such dynamic graphs for better stock predictions, we propose the hybrid-attention dynamic graph neural network, an inductive graph representation learning method. We also extended mini-batch gradient descent to inductive representation learning on dynamic stock graphs so that the model can update parameters over mini-batch stock graphs with higher training efficiency. Extensive experiments on stocks from different markets and trading simulations demonstrate that the proposed method significantly improves stock predictions. The proposed method can have important implications for the management of financial portfolios and investment risk. Summary of Contribution: Accurate predictions of stock prices have important implications for financial decisions. In today’s economy, individual firms are increasingly connected via different types of relationships. As a result, firms’ stock prices often feature synchronous co-movement patterns. This paper represents the first effort to leverage such phenomena to construct dynamic stock graphs for stock predictions. We develop hybrid-attention dynamic graph neural network (HAD-GNN), an inductive graph representation learning framework for dynamic stock graphs to incorporate temporal and graph attention mechanisms. To improve the learning efficiency of HAD-GNN, we also extend the mini-batch gradient descent to inductive representation learning on such dynamic graphs and adopt a t-batch training mechanism (t-BTM). We demonstrate the effectiveness of our new approach via experiments based on real-world data and simulations.

Inductive Graph Representation Learning for fraud detection

Graphs can be seen as a universal language to describe and model a diverse set of complex systems and data structures. However, efficiently extracting topological information from dynamic graphs is not a straightforward task. Previous works have explored a variety of inductive graph representation learning frameworks, but despite the surge in development, little research deployed these techniques for real-life applications. Most earlier studies are restricted to a set of benchmark experiments, rendering their practical generalisability questionable. This paper evaluates the proclaimed predictive performance of state-of-the-art inductive graph representation learning algorithms on highly imbalanced credit card transaction networks. More specifically, we assess the inductive capability of GraphSAGE and Fast Inductive Graph Representation Learning in a fraud detection setting. Credit card transaction fraud networks pose two crucial challenges for graph representation learners: First, these networks are highly dynamic, continuously encountering new transactions. Second, they are heavily imbalanced, with only a small fraction of transactions labelled as fraudulent. This paper contributes to the literature by (i) proving how inductive graph representation learning techniques can be leveraged to enhance predictive performance for fraud detection and (ii) demonstrating the benefit of graph-level undersampling for representation learning in imbalanced networks.

Inductive Spatiotemporal Graph Convolutional Networks for Short-Term Quantitative Precipitation Forecasting

Short-term quantitative precipitation forecasting (SQPF) using weather radar is an important but challenging problem as one must cope with inherent nonlinearity and spatiotemporal correlation in the data. In this article, we propose a novel deep learning model, named Inductive spatiotemporal Graph Convolutional Networks (InstGCNs), to overcome these issues in SQPF. The proposed InstGCN can learn a nonlinear mapping from historical radar reflectivity to future rainfall amounts and extract informative spatiotemporal representations simultaneously. Specifically, we first provide a formal definition for formulating the SQPF problem from a graph perspective. Then, based on radar reflectivity and rain gauge observation, we propose a novel graph construction approach that utilizes a special elliptic structure to model the spatial dependence of precipitation areas. In addition, a new Node level Differential Block (Node-DB) is introduced to tackle the nonstationary temporal dependence. To execute inductive graph learning for unseen nodes, we design to decompose a whole graph into subgraphs. We conduct extensive experiments on three real-world datasets in East China and a public weather radar dataset in the southeastern parts of France. The experimental results confirm the advantages of InstGCN compared with several state of the arts.