Graph Convolutional Layers Research Articles

Multi-view contrastive clustering via integrating graph aggregation and confidence enhancement

Multi-view clustering endeavors to effectively uncover consistent clustering patterns across multiple data sources or feature spaces. This field grapples with two key challenges: (1) the effective integration and utilization of consistency and complementarity information from diverse view spaces, and (2) the capturing of structural correlations between data samples in the multi-view context. To address these challenges, this paper proposes the Multi-view contrAstive clustering with Graph Aggregation and confidence enhancement (MAGA) algorithm. Specifically, we employ a deep autoencoder network to learn embedded features for each independent view. To harness consistency and complementarity information, we introduce the Simple Cross-view Spectral Graph Aggregation module. This module utilizes graph convolutional layers to generate view-specific graph embeddings and subsequently aggregates these embeddings from different views into a unified feature space using a cross-view self-attention mechanism. To capture both inter-view and intra-view structural correlations among different samples, we propose a dual representation contrastive learning mechanism, which operates concurrently at both the instance and feature levels. Additionally, we introduce the maximizing cluster assignment confidence mechanism to obtain more compact clustering assignments. As a result, MAGA outperforms 20 competitive methods across nine benchmark datasets, showcasing its superior performance. Code: https://github.com/BJT-bjt/MAGA.

GraphMHC: Neoantigen prediction model applying the graph neural network to molecular structure.

Neoantigens are tumor-derived peptides and are biomarkers that can predict prognosis related to immune checkpoint inhibition by estimating their binding to major histocompatibility complex (MHC) proteins. Although deep neural networks have been primarily used for these prediction models, it is difficult to interpret the models reported thus far as accurately representing the interactions between biomolecules. In this study, we propose the GraphMHC model, which utilizes a graph neural network model applied to molecular structure to simulate the binding between MHC proteins and peptide sequences. Amino acid sequences sourced from the immune epitope database (IEDB) undergo conversion into molecular structures. Subsequently, atomic intrinsic informations and inter-atomic connections are extracted and structured as a graph representation. Stacked graph attention and convolution layers comprise the GraphMHC network which classifies bindings. The prediction results from the test set using the GraphMHC model showed a high performance with an area under the receiver operating characteristic curve of 92.2% (91.9-92.5%), surpassing a baseline model. Moreover, by applying the GraphMHC model to melanoma patient data from The Cancer Genome Atlas project, we found a borderline difference (0.061) in overall survival and a significant difference in stromal score between the high and low neoantigen load groups. This distinction was not present in the baseline model. This study presents the first feature-intrinsic method based on biochemical molecular structure for modeling the binding between MHC protein sequences and neoantigen candidate peptide sequences. This model can provide highly accurate responsibility information that can predict the prognosis of immune checkpoint inhibitors to cancer patients who want to apply it.

Residual Hyperbolic Graph Convolution Networks

Hyperbolic graph convolutional networks (HGCNs) have demonstrated representational capabilities of modeling hierarchical-structured graphs. However, as in general GCNs, over-smoothing may occur as the number of model layers increases, limiting the representation capabilities of most current HGCN models. In this paper, we propose residual hyperbolic graph convolutional networks (R-HGCNs) to address the over-smoothing problem. We introduce a hyperbolic residual connection function to overcome the over-smoothing problem, and also theoretically prove the effectiveness of the hyperbolic residual function. Moreover, we use product manifolds and HyperDrop to facilitate the R-HGCNs. The distinctive features of the R-HGCNs are as follows: (1) The hyperbolic residual connection preserves the initial node information in each layer and adds a hyperbolic identity mapping to prevent node features from being indistinguishable. (2) Product manifolds in R-HGCNs have been set up with different origin points in different components to facilitate the extraction of feature information from a wider range of perspectives, which enhances the representing capability of R-HGCNs. (3) HyperDrop adds multiplicative Gaussian noise into hyperbolic representations, such that perturbations can be added to alleviate the over-fitting problem without deconstructing the hyperbolic geometry. Experiment results demonstrate the effectiveness of R-HGCNs under various graph convolution layers and different structures of product manifolds.

MSGNet: Learning Multi-Scale Inter-series Correlations for Multivariate Time Series Forecasting

Multivariate time series forecasting poses an ongoing challenge across various disciplines. Time series data often exhibit diverse intra-series and inter-series correlations, contributing to intricate and interwoven dependencies that have been the focus of numerous studies. Nevertheless, a significant research gap remains in comprehending the varying inter-series correlations across different time scales among multiple time series, an area that has received limited attention in the literature. To bridge this gap, this paper introduces MSGNet, an advanced deep learning model designed to capture the varying inter-series correlations across multiple time scales using frequency domain analysis and adaptive graph convolution. By leveraging frequency domain analysis, MSGNet effectively extracts salient periodic patterns and decomposes the time series into distinct time scales. The model incorporates a self-attention mechanism to capture intra-series dependencies, while introducing an adaptive mixhop graph convolution layer to autonomously learn diverse inter-series correlations within each time scale. Extensive experiments are conducted on several real-world datasets to showcase the effectiveness of MSGNet. Furthermore, MSGNet possesses the ability to automatically learn explainable multi-scale inter-series correlations, exhibiting strong generalization capabilities even when applied to out-of-distribution samples.

LSTTN: A Long-Short Term Transformer-based spatiotemporal neural network for traffic flow forecasting

Accurate traffic forecasting is a fundamental problem in intelligent transportation systems and learning long-range traffic representations with key information through spatiotemporal graph neural networks (STGNNs) is a basic assumption of current traffic flow prediction models. However, due to structural limitations, existing STGNNs can only utilize short-range traffic flow data; therefore, the models cannot adequately learn the complex trends and periodic features in traffic flow. Besides, it is challenging to extract the key temporal information from the long historical traffic series and obtain a compact representation. To solve the above problems, we propose a novel LSTTN (Long-Short Term Transformer-based Network) framework comprehensively considering the long- and short-term features in historical traffic flow. First, we employ a masked subseries Transformer to infer the content of masked subseries from a small portion of unmasked subseries and their temporal context in a pretraining manner, forcing the model to efficiently learn compressed and contextual subseries temporal representations from long historical series. Then, based on the learned representations, long-term trend is extracted by using stacked 1D dilated convolution layers, and periodic features are extracted by dynamic graph convolution layers. For the difficulties in making time-step level prediction, LSTTN adopts a short-term trend extractor to learn fine-grained short-term temporal features. Finally, LSTTN fuses the long-term trend, periodic features and short-term features to obtain the prediction results. Experiments on four real-world datasets show that in 60-minute-ahead long-term forecasting, the LSTTN model achieves a minimum improvement of 5.63% and a maximum improvement of 16.78% over baseline models. The source code is availble at https://github.com/GeoX-Lab/LSTTN.

Robust Graph Autoencoder-Based Detection of False Data Injection Attacks Against Data Poisoning in Smart Grids

Machine learning-based detection of false data injection attacks (FDIAs) in smart grids relies on labeled measurement data for training and testing. The majority of existing detectors are developed assuming that the adopted datasets for training have correct labeling information. However, such an assumption is not always valid as training data might include measurement samples that are incorrectly labeled as benign, namely, adversarial data poisoning samples, which have not been detected before. Neglecting such an aspect makes detectors susceptible to data poisoning. Our investigations revealed that detection rates (DRs) of existing detectors significantly deteriorate by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$9 - 29\%$</tex-math></inline-formula> when subject to data poisoning in generalized and topology-specific settings. Thus, we propose a generalized graph neural network (GNN)-based anomaly detector that is robust against FDIAs and data poisoning. It requires only benign datasets for training and employs an autoencoder with Chebyshev graph convolutional recurrent layers with attention mechanism to capture the spatial and temporal correlations within measurement data. The proposed convolutional recurrent graph autoencoder (CR-GAE) model is trained and tested on various topologies (from 14, 39, and 118-bus systems). Due to such factors, it yields stable generalized detection performance that is degraded by only <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1.6 - 3.7\%$</tex-math></inline-formula> in DR against high levels of data poisoning and unseen FDIAs in unobserved topologies.

Spatial-spectral graph convolutional extreme learning machine for hyperspectral image classification

ABSTRACT Hyperspectral image has excellent spectral information and abundant spatial information, and its feature quality is one of the key factors affecting the classification performance. Extreme learning machine (ELM) is widely used in hyperspectral classification problems. However, traditional ELM is usually based on regular Euclidean data, ignoring the inherent structured information between hyperspectral pixels, resulting in poor robustness. In this paper, we propose a new supervised extreme learning machine framework, termed spatial-spectral graph convolutional ELM (SSG-ELM), based on using graph convolution to extend ELM into the non-Euclidean domain. The method inherits all the advantages from ELM, and consists of a random graph convolutional layer followed by a graph convolutional regression layer, enabling it to model complex intra class variations. Otherwise, a local spectral-spatial context integration and reshaping mechanism is incorporated into the hidden layer feature representation by using a context-aware bilateral filtering procedure. Experimental results show that the proposed algorithm obtains a competitive performance and outperforms other state-of-the-art ELM-based methods.

Spatio-temporal neural network with handcrafted features for skeleton-based action recognition

The task of human action recognition (HAR) can be found in many computer vision practical applications. Various data modalities have been considered for solving this task, including joint-based skeletal representations which are suitable for real-time applications on platforms with limited computational resources. We propose a spatio-temporal neural network that uses handcrafted geometric features to classify human actions from video data. The proposed deep neural network architecture combines graph convolutional and temporal convolutional layers. The experiments performed on public HAR datasets show that our model obtains results similar to other state-of-the-art methods but has a lower inference time while offering the possibility to obtain an explanation for the classified action.

Prediction of multiple types of drug interactions based on multi-scale fusion and dual-view fusion.

Potential drug-drug interactions (DDI) can lead to adverse drug reactions (ADR), and DDI prediction can help pharmacy researchers detect harmful DDI early. However, existing DDI prediction methods fall short in fully capturing drug information. They typically employ a single-view input, focusing solely on drug features or drug networks. Moreover, they rely exclusively on the final model layer for predictions, overlooking the nuanced information present across various network layers. To address these limitations, we propose a multi-scale dual-view fusion (MSDF) method for DDI prediction. More specifically, MSDF first constructs two views, topological and feature views of drugs, as model inputs. Then a graph convolutional neural network is used to extract the feature representations from each view. On top of that, a multi-scale fusion module integrates information across different graph convolutional layers to create comprehensive drug embeddings. The embeddings from the two views are summed as the final representation for classification. Experiments on two real-world datasets demonstrate that MSDF achieves higher accuracy than state-of-the-art methods, as the dual-view, multi-scale approach better captures drug characteristics.

A graph neural network based fault diagnosis strategy for power communication networks

ABSTRACT The power communication network (PCN) is the backbone network of the power system. However, faulty nodes in the network may cause communication interruptions, seriously affecting the reliable operation of the power system. Therefore, accurate diagnosis of faulty nodes is crucial for timely detection, localization, and troubleshooting. Based on graph neural networks, we propose a Neighbour Selection and Merge Fault Diagnosis (NSMFD) strategy, which aims to identify faulty nodes capturing graph structure information and node feature information. First, we construct a graph representation of PCN, where nodes represent devices in the network and edges represent the connections between devices. Then, we use sampling and aggregation in node embedding to capture adjacent feature information of nodes, gradually fuse and update node representations through graph convolutional layers, which could be applied as the input layer of diagnostic process. Finally, we use softmax and cross entropy loss function to get a probabilistic representation and optimize it by backpropagation for prediction. We conduct experiments using real PCN datasets and compare it with other advanced diagnostic methods in terms of accuracy, false positive rate, and false negative rate. Compared with diagnostic methods, our method can accurately identify faulty nodes and achieve timely fault detection and recovery.

Learning Embedding for Signed Network in Social Media With Global Information

Signed networks contain nodes connected by positive and negative signed links. Signed network representation learning concentrates on learning the low-dimensional representations of these nodes, which facilitates downstream tasks such as link prediction using general data mining framework. However, most signed network embedding (SiNE) approaches neglect global information in the networks, limiting the capacity to learn genuine signed graph topology. To overcome this limitation, a deep graph neural network (GNN) framework SiG to learn SiNE with global information is proposed. To be more explicit, a hierarchical pooling mechanism is developed to encode the high-level features of the networks. Moreover, a graph convolution layer is introduced to aggregate both positive and negative information from neighbor nodes, and the concatenation of two parts generates the final embedding of nodes. Extensive experiments on four large real-world signed network datasets demonstrate the effectiveness and excellence of the proposed method.

SSFG: Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networks.

Graph convolutional networks (GCNs) have been successfully applied in various graph-based tasks. In a typical graph convolutional layer, node features are updated by aggregating neighborhood information. Repeatedly applying graph convolutions can cause the oversmoothing issue, i.e., node features at deep layers converge to similar values. Previous studies have suggested that oversmoothing is one of the major issues that restrict the performance of GCNs. In this article, we propose a stochastic regularization method to tackle the oversmoothing problem. In the proposed method, we stochastically scale features and gradients (SSFG) by a factor sampled from a probability distribution in the training procedure. By explicitly applying a scaling factor to break feature convergence, the oversmoothing issue is alleviated. We show that applying stochastic scaling at the gradient level is complementary to that applied at the feature level to improve the overall performance. Our method does not increase the number of trainable parameters. When used together with ReLU, our SSFG can be seen as a stochastic ReLU activation function. We experimentally validate our SSFG regularization method on three commonly used types of graph networks. Extensive experimental results on seven benchmark datasets for four graph-based tasks demonstrate that our SSFG regularization is effective in improving the overall performance of the baseline graph networks. The code is available at https://github.com/vailatuts/SSFG-regularization.

Less is More: Removing Redundancy of Graph Convolutional Networks for Recommendation

While Graph Convolutional Networks (GCNs) have shown great potential in recommender systems and collaborative filtering (CF), they suffer from expensive computational complexity and poor scalability. On top of that, recent works mostly combine GCNs with other advanced algorithms which further sacrifice model efficiency and scalability. In this work, we unveil the redundancy of existing GCN-based methods in three aspects: (1) Feature redundancy . By reviewing GCNs from a spectral perspective, we show that most spectral graph features are noisy for recommendation, while stacking graph convolution layers can suppress but cannot completely remove the noisy features, which we mostly summarize from our previous work; (2) Structure redundancy . By providing a deep insight into how user/item representations are generated, we show that what makes them distinctive lies in the spectral graph features, while the core idea of GCNs (i.e., neighborhood aggregation) is not the reason making GCNs effective; and (3) Distribution redundancy . Following observations from (1), we further show that the number of required spectral features is closely related to the spectral distribution, where important information tends to be concentrated in more (fewer) spectral features on a flatter (sharper) distribution. To make important information be concentrated in as few features as possible, we sharpen the spectral distribution by increasing the node similarity without changing the original data, thereby reducing the computational cost. To remove these three kinds of redundancies, we propose a Simplified Graph Denoising Encoder (SGDE) only exploiting the top- K singular vectors without explicitly aggregating neighborhood, which significantly reduces the complexity of GCN-based methods. We further propose a scalable contrastive learning framework to alleviate data sparsity and to boost model robustness and generalization, leading to significant improvement. Extensive experiments on three real-world datasets show that our proposed SGDE not only achieves state-of-the-art but also shows higher scalability and efficiency than our previously proposed GDE as well as traditional and GCN-based CF methods.

A Quantum Spatial Graph Convolutional Neural Network Model on Quantum Circuits.

This article proposes a quantum spatial graph convolutional neural network (QSGCN) model that is implementable on quantum circuits, providing a novel avenue to processing non-Euclidean type data based on the state-of-the-art parameterized quantum circuit (PQC) computing platforms. Four basic blocks are constructed to formulate the whole QSGCN model, including the quantum encoding, the quantum graph convolutional layer, the quantum graph pooling layer, and the network optimization. In particular, the trainability of the QSGCN model is analyzed through discussions on the barren plateau phenomenon. Simulation results from various types of graph data are presented to demonstrate the learning, generalization, and robustness capabilities of the proposed quantum neural network (QNN) model.

Graph Neural Network-Based EEG Classification: A Survey.

Graph neural networks (GNN) are increasingly used to classify EEG for tasks such as emotion recognition, motor imagery and neurological diseases and disorders. A wide range of methods have been proposed to design GNN-based classifiers. Therefore, there is a need for a systematic review and categorisation of these approaches. We exhaustively search the published literature on this topic and derive several categories for comparison. These categories highlight the similarities and differences among the methods. The results suggest a prevalence of spectral graph convolutional layers over spatial. Additionally, we identify standard forms of node features, with the most popular being the raw EEG signal and differential entropy. Our results summarise the emerging trends in GNN-based approaches for EEG classification. Finally, we discuss several promising research directions, such as exploring the potential of transfer learning methods and appropriate modelling of cross-frequency interactions.

Multi-Scale FC-Based Multi-Order GCN: A Novel Model for Predicting Individual Behavior From fMRI.

Predicting individual behavior from brain imaging data using machine learning is a rapidly growing field in neuroscience. Functional connectivity (FC), which captures interactions between different brain regions, contains valuable information about the organization of the brain and is considered a crucial feature for modeling human behavior. Graph convolutional networks (GCN) have proven to be a powerful tool for extracting graph structure features and have shown promising results in various FC-based classification tasks, such as disease classification and prognosis prediction. Despite this success, few behavior prediction models currently exist based on GCN, and their performance is not satisfactory. To address this gap, a new model called the Multi-Scale FC-based Multi-Order GCN (MSFC-MO-GCN) was proposed in this paper. The model considers the hierarchical structure of the brain system and utilizes FCs inferred from multiple spatial scales as input to comprehensively characterize individual brain organization. To enhance the feature learning ability of GCN, a multi-order graph convolutional layer is incorporated, which uses multi-order neighbors to guide message passing and learns high-order graph information of nodal connections. Additionally, an inter-subject contrast constraint is designed to control the potential information redundancy of FCs among different spatial scales during the feature learning process. Experimental evaluation were conducted on the publicly available dataset from human connectome project. A total of 805 healthy subjects were included and 5 representative behavior metrics were used. The experimental results show that our proposed method outperforms the existing behavior prediction models in all behavior prediction tasks.

Spatio-Temporal Joint Graph Convolutional Networks for Traffic Forecasting

Recent studies have shifted their focus towards formulating traffic forecasting as a spatio-temporal graph modeling problem. Typically, they constructed a static spatial graph at each time step and then connected each node with itself between adjacent time steps to create a spatio-temporal graph. However, this approach failed to explicitly reflect the correlations between different nodes at different time steps, thus limiting the learning capability of graph neural networks. Additionally, those models overlooked the dynamic spatio-temporal correlations among nodes by using the same adjacency matrix across different time steps. To address these limitations, we propose a novel approach called Spatio-Temporal Joint Graph Convolutional Networks (STJGCN) for accurate traffic forecasting on road networks over multiple future time steps. Specifically, our method encompasses the construction of both pre-defined and adaptive spatio-temporal joint graphs (STJGs) between any two time steps, which represent comprehensive and dynamic spatio-temporal correlations. We further introduce dilated causal spatio-temporal joint graph convolution layers on the STJG to capture spatio-temporal dependencies from distinct perspectives with multiple ranges. To aggregate information from different ranges, we propose a multi-range attention mechanism. Finally, we evaluate our approach on five public traffic datasets and experimental results demonstrate that STJGCN is not only computationally efficient but also outperforms 11 state-of-the-art baseline methods.

Residual-aware deep attention graph convolutional network via unveiling data latent interactions for product quality prediction in industrial processes

Data-driven modeling based on deep learning is crucial for online product quality prediction in industrial processes. Extracting latent data interactions from sensor variables lies at the core of various data-driven modeling applications. Typically, the observed variables exhibit non-stationary properties due to changes in operating conditions and sensor tuning issues. These fluctuations inevitably affect the reliability of traditional feature extraction methods, thus hindering their applications. Hence, this paper proposes a novel method named residual-aware deep attention graph convolutional network (RaDA-GCN) to explore potential interactions among sensor variables. RaDA-GCN ingeniously incorporates an attention mechanism into the graph convolution layer to extract nonlinear variable-related features according to their importance. Then, a novel residual-aware connection module is designed to reduce data uncertainty and alleviate over-smoothing. By skillfully stacking multiple attention map convolutional layers with the integration of residual-aware connections, deep structural features are obtained to facilitate the effective quantification and revelation of potential relationships among data variables. Finally, the application of the predictive modeling framework based on the proposed method is used to validate its effectiveness in actual industrial process data. The experimental results demonstrate that the proposed RaDA-GCN method exhibits a 23% improvement in the R-squared (R2) indicator and a 13% reduction in the root mean square error (RMSE) compared to the traditional graph convolutional network method.

A unified framework for backpropagation-free soft and hard gated graph neural networks

We propose a framework for the definition of neural models for graphs that do not rely on backpropagation for training, thus making learning more biologically plausible and amenable to parallel implementation. Our proposed framework is inspired by Gated Linear Networks and allows the adoption of multiple graph convolutions. Specifically, each neuron is defined as a set of graph convolution filters (weight vectors) and a gating mechanism that, given a node and its topological context, generates the weight vector to use for processing the node’s attributes. Two different graph processing schemes are studied, i.e., a message-passing aggregation scheme where the gating mechanism is embedded directly into the graph convolution, and a multi-resolution one where neighboring nodes at different topological distances are jointly processed by a single graph convolution layer. We also compare the effectiveness of different alternatives for defining the context function of a node, i.e., based on hyperplanes or on prototypes, and using a soft or hard-gating mechanism. We propose a unified theoretical framework allowing us to theoretically characterize the proposed models’ expressiveness. We experimentally evaluate our backpropagation-free graph convolutional neural models on commonly adopted node classification datasets and show competitive performances compared to the backpropagation-based counterparts.

Graph t-SNE multi-view autoencoder for joint clustering and completion of incomplete multi-view data

In recent years, incomplete multi-view clustering has attracted much attention and achieved promising performances through the use of deep learning. However, only a few prior methods are concerned with joint missing data recovery and clustering. In this paper, we present a graph t-SNE multi-view autoencoder (GTSNE-MvAE) for this task. We formulate the view completion problem as a multi-view multivariate regression and reconsider the autoencoder for this task. First, a multi-view encoder augmented with graph-convolutional layers and the t-SNE regularization loss extracts unified representations from incomplete multi-view features. Then, the representations are fed into view-specific decoders to regress the features of views, through which the missing views are recovered. Notably, the unified representations learned by our model are cluster-friendly. Our simple method achieves competitive clustering performances on 9 challenging public benchmarks while keeping a stable training process and hyperparameter insensitivity.