Multi-scale Graph Convolutional Network Research Articles

Innovative graph neural network approach for predicting soil heavy metal pollution in the Pearl River Basin, China

Predicting soil heavy metal (HM) content is crucial for monitoring soil quality and ensuring ecological health. However, existing methods often neglect the spatial dependency of data. To address this gap, our study introduces a novel graph neural network (GNN) model, Multi-Scale Attention-based Graph Neural Network for Heavy Metal Prediction (MSA-GNN-HMP). The model integrates multi-scale graph convolutional network (MS-GCN) and attention-based GNN (AGNN) to capture spatial relationships. Using surface soil samples from the Pearl River Basin, we evaluate the MSA-GNN-HMP model against four other models. The experimental results show that the MSA-GNN-HMP model has the best predictive performance for Cd and Pb, with a coefficient of determination (R2) of 0.841 for Cd and 0.886 for Pb, and the lowest mean absolute error (MAE) of 0.403 mg kg−1 for Cd and 0.670 mg kg−1 for Pb, as well as the lowest root mean square error (RMSE) of 0.563 mg kg−1for Cd and 0.898 mg kg−1 for Pb. In feature importance analysis, latitude and longitude emerged as key factors influencing the heavy metal content. The spatial distribution prediction trend of heavy metal elements by different prediction methods is basically consistent, with the high-value areas of Cd and Pb respectively distributed in the northwest and northeast of the basin center. However, the MSA-GNN-HMP model demonstrates superior detail representation in spatial prediction. MSA-GNN-HMP model has excellent spatial information representation capabilities and can more accurately predict heavy metal content and spatial distribution, providing a new theoretical basis for monitoring, assessing, and managing soil pollution.

Semi-Supervised Multiscale Dynamic Graph Convolution Network for Hyperspectral Image Classification.

In recent years, convolutional neural networks (CNNs)-based methods achieve cracking performance on hyperspectral image (HSI) classification tasks, due to its hierarchical structure and strong nonlinear fitting capacity. Most of them, however, are supervised approaches that need a large number of labeled data to train them. Conventional convolution kernels are fixed shape of rectangular with fixed sizes, which are good at capturing short-range relations between pixels within HSIs but ignore the long-range context within HSIs, limiting their performance. To overcome the limitations mentioned above, we present a dynamic multiscale graph convolutional network (GCN) classifier (DMSGer). DMSGer first constructs a relatively small graph at region-level based on a superpixel segmentation algorithm and metric-learning. A dynamic pixel-level feature update strategy is then applied to the region-level adjacency matrix, which can help DMSGer learn the pixel representation dynamically. Finally, to deeply understand the complex contents within HSIs, our model is expanded into a multiscale version. On the one hand, by introducing graph learning theory, DMSGer accomplishes HSI classification tasks in a semi-supervised manner, relieving the pressure of collecting abundant labeled samples. Superpixels are generally in irregular shapes and sizes which can group only similar pixels in a neighborhood. On the other hand, based on the proposed dynamic-GCN, the pixel-level and region-level information can be captured simultaneously in one graph convolution layer such that the classification results can be improved. Also, due to the proper multiscale expansion, more helpful information can be captured from HSIs. Extensive experiments were conducted on four public HSIs, and the promising results illustrate that our DMSGer is robust in classifying HSIs. Our source codes are available at https://github.com/TangXu-Group/DMSGer.

A multi-scale graph convolutional network with contrastive-learning enhanced self-attention pooling for intelligent fault diagnosis of gearbox

Recently, the emerging graph convolutional network (GCN) has been applied into fault diagnosis with the aim of providing additional fault features through topological information. However, there are some limitations with these methods. First, the interactions between multi-frequency scales are ignored in existing studies, while they mainly focus on constructing graphs through the relationship between channels/instances. Second, the constructed graph cannot well reflect the topology of noisy samples and lacks robust hierarchical representation learning capability, and the learned graphs have limited interpretability. Hence, a Multi-Scale GCN with Contrastive-learning enhanced Self-attention Pooling (MSGCN-CSP) method is proposed for intelligent fault diagnosis of gearbox. Time–frequency distributions are converted into multi-scale graphs to extract fault features through topological relationships between multi-frequencies. Contrastive-learning is used to implement graph pooling, which enables hierarchical representation learning. Experimental results on two gearbox datasets illustrate that the proposed method offers competitive diagnostic performance and provides good interpretability in establishing GCN.

Dense Multi-Scale Graph Convolutional Network for Knee Joint Cartilage Segmentation.

In this paper, we propose a dense multi-scale adaptive graph convolutional network (DMA-GCN) method for automatic segmentation of the knee joint cartilage from MR images. Under the multi-atlas setting, the suggested approach exhibits several novelties, as described in the following. First, our models integrate both local-level and global-level learning simultaneously. The local learning task aggregates spatial contextual information from aligned spatial neighborhoods of nodes, at multiple scales, while global learning explores pairwise affinities between nodes, located globally at different positions in the image. We propose two different structures of building models, whereby the local and global convolutional units are combined by following an alternating or a sequential manner. Secondly, based on the previous models, we develop the DMA-GCN network, by utilizing a densely connected architecture with residual skip connections. This is a deeper GCN structure, expanded over different block layers, thus being capable of providing more expressive node feature representations. Third, all units pertaining to the overall network are equipped with their individual adaptive graph learning mechanism, which allows the graph structures to be automatically learned during training. The proposed cartilage segmentation method is evaluated on the entire publicly available Osteoarthritis Initiative (OAI) cohort. To this end, we have devised a thorough experimental setup, with the goal of investigating the effect of several factors of our approach on the classification rates. Furthermore, we present exhaustive comparative results, considering traditional existing methods, six deep learning segmentation methods, and seven graph-based convolution methods, including the currently most representative models from this field. The obtained results demonstrate that the DMA-GCN outperforms all competing methods across all evaluation measures, providing DSC=95.71% and DSC=94.02% for the segmentation of femoral and tibial cartilage, respectively.

Difference-guided multiscale graph convolution network for unsupervised change detection in PolSAR images

Image change detection is important in polarimetric synthetic aperture radar (PolSAR) image analysis and interpretation. However, improving its accuracy is challenging because of the interference of multiplicative speckle noise. To address this issue, we propose an unsupervised PolSAR image change detection method based on a multiscale graph convolutional network (GCN). First, a Shannon entropy difference image is introduced and improved to obtain an enhanced difference image (EDI) that can effectively suppress speckle noise while preserving edge information. The generated EDI can be further utilised to construct a pseudo-label set required for unsupervised change detection. Subsequently, a difference constraint joint graph construction (DCJGC) module is proposed to obtain the object-level input information of the network. This uses the joint superpixels of multitemporal PolSAR images as graph nodes, and then introduces the difference information in the EDI to constrain the formation process of the edges between the nodes, efficiently and accurately constructing undirected graphs. Finally, a difference-guided multiscale GCN (DGMGCN) is designed for PolSAR image change detection. The network utilises difference information to eliminate the adverse effect of speckle noise on change detection and fully capture the change-aware features of multitemporal PolSAR images at fine and coarse scales, thereby improving feature discriminability. Experimental results on six real Gaofen-3 PolSAR datasets validate the superiority of the proposed approach over other state-of-the-art methods.

Hierarchical parallel multi-scale graph network for 3d human pose estimation

Graph convolutional network (GCN) is a widespread architecture for 2D-to-3D human pose estimation (HPE). Vanilla graph convolution is the key in GCN for body joints feature extraction by aggregating features from the first-order neighbors of each joint at single-scale. However, the nodes features updated in graph are gradually assimilated with the network deepens This makes it difficult to model the powerful feature representation for joints and has an adverse effect on resolving the uncertainty caused by occlusion or depth ambiguity. To address these problems, we propose a hierarchical parallel multi-scale graph convolutional network (HPM-GNet) for 3D HPE in this paper. The proposed network is composed by multi-scale sub-graph networks in parallel framework, guided by the geometric constraint of human body. Firstly, a well-designed weight masked graph convolutional (WMGConv) layer is used as a fundamental unit to construct the parallel multi-scale sub-graph convolutional network module (PMGCN) in HPM-GNet, which corporately captures the features of target nodes and neighbor nodes with weight assignment. Then, a cross-scale features exchange fusion block (CFEB) is designed to aggregate multi-scale features from local and global perspective with considering the geometric information from different parts of human body. Finally, we conduct experiments on two challenging 3D human pose benchmark datasets to evaluate the effectiveness of the proposed model. The experimental results demonstrate that our model achieves state-of-the-art performance.

Anomaly detection based on joint spatio-temporal learning for building electricity consumption

The use of electric energy is an integral part of people's daily life. Anomaly detection of electricity consumption data, as a classification problem, has always been a hot research topic of scholars. Anomaly detection can not only reduce energy waste, but also prevent small problems from becoming overwhelming problems. At present, most anomaly detection algorithms mainly focus on the time series information of electricity consumption data, while ignoring the spatial feature of electricity consumption data. To fill this research gap, the paper proposes an Anomaly Detection based on Joint Spatio-Temporal learning (ADJST) method for building electricity consumption. First, a Multi-Scale Graph Convolutional Network (MS-GCN) is proposed to learn the spatial features of building electricity consumption data. Specifically, two types of graphs are constructed to extract short-term correlation features and long-term regularity features of building electricity consumption data. Second, a Multi-Scale Temporal Convolutional Network (MS-TCN) is proposed to learn the temporal features of building electricity consumption data. Adopt a multi-scale vanilla convolution structure to extract multi-scale time series information from building electricity consumption data. Third, the combination of temporal features and spatial features detects anomalous electricity consumption of marked users. Final, taken the user electricity consumption data collected by the State Grid Corporation's smart meter as examples, compared with a variety of classical anomaly detection algorithms, the results of F1-score and AUC of the proposed method are 0.935 and 0.977 respectively, which proves the superiority of the method. The model shows good stability in dealing with extreme imbalance of data, and is proved to be generalized by experiments and can be transferred to other datasets.

Multiscale Wavelet-Driven Graph Convolutional Network for Blade Icing Detection of Wind Turbines

Blade icing detection is critical to maintaining the health of wind turbines, especially in cold climates. Rapid and accurate icing detection allows proper control of wind turbines, including shutting down and clearing the ice, thus ensuring turbine safety. This article presents a wavelet-driven multiscale graph convolutional network (MWGCN), which is a supervised deep learning model for blade icing detection. The proposed model first uses wavelet decomposition to capture multivariate information in the time and frequency domains, and then employs a temporal graph convolutional network (GCN) to model the intervariable correlations of the decomposed multiscale wavelets and their temporal dynamics. In addition, this article introduces scale attention to the MWGCN for a further improvement of the model and proposes the method to address the class imbalance problem of the training data sets. Finally, the article conducts comprehensive experiments to evaluate the proposed model, and the results demonstrate the effectiveness of the model in blade icing detection and its better performance over eight state-of-the-art algorithms, with 17.2% and 11.3% higher F1 scores over the best state-of-the-art baseline on the labeled datasets.

Multiscale Graph-Guided Convolutional Network With Node Attention for Intelligent Health State Diagnosis of a 3-PRR Planar Parallel Manipulator

The data-driven intelligent health state diagnosis strategy has been successfully applied in modern industrial equipments. However, many of existing research works suffer from two major deficiencies: First, the sample independence assumption is widely adopted, so the influences of the relationship between samples on the overall performances are not explored. Second, most of the abovementioned application objects are typical key functional components (such as bearings, gearboxes, etc.), and research works on the intelligent health state diagnosis of planar parallel manipulator are rarely reported. To address these issues, a novel intelligent health state diagnosis method, termed multiscale graph-guided convolutional network with node attention (MSGCN-NA), is proposed for a 3-PRR (P and R represent prismatic and revolute pairs, respectively) planar parallel manipulator. Specifically, the developed MSGCN-NA model mainly contains the following two parts: first, the one is an unsupervised convolutional autoencoder, which is employed for the extraction of deep representation features, and then combined with Pearson metric to establish the adjacency matrix. Second, The other part is the constructed multiscale graph convolutional network, in which the node attention mechanism is adopted to achieve cross-scale fusion of different neighborhood information. The effectiveness of the proposed MSGCN-NA method is fully verified based on the simulation and experimental scenarios, the results show that MSGCN-NA can achieve superior diagnosis performances.

A Novel Hybrid Attention-Driven Multistream Hierarchical Graph Embedding Network for Remote Sensing Object Detection

Multiclass geospatial object detection in high-spatial-resolution remote-sensing images (HSRIs) has recently attracted considerable attention in many remote-sensing applications as a fundamental task. However, the complexity and uncertainty of spatial distribution among multiclass geospatial objects are still huge challenges for object detection in HSRIs. Most current remote-sensing object-detection approaches fall back on deep convolutional neural networks (CNNs). Nevertheless, most existing methods only focus on mining visual characteristics and lose sight of spatial or semantic relation discriminations, eventually degrading object-detection performance in HSRIs. To tackle these challenges, we propose a novel hybrid attention-driven multistream hierarchical graph embedding network (HA-MHGEN) to explore complementary spatial and semantic patterns for improving remote-sensing object-detection performance. Specifically, we first constructed hierarchical spatial graphs for multiscale spatial relation representation. Then, semantic graphs were also constructed by integrating them with the word embedding of object category labels on graph nodes. Afterwards, we developed a self-attention-aware multiscale graph convolutional network (GCN) to derive stronger for intra- and interobject hierarchical spatial relations and contextual semantic relations, respectively. These two relation networks were followed by a novel cross-attention-driven spatial- and semantic-feature fusion module that utilizes a multihead attention mechanism to learn associations between diverse spatial and semantic correlations, and guide them to endowing a more powerful discrimination ability. With the collaborative learning of the three relation networks, the proposed HA-MHGEN enables grasping explicit and implicit relations from spatial and semantic patterns, and boosts multiclass object-detection performance in HRSIs. Comprehensive and extensive experimental evaluation results on three benchmarks, namely, DOTA, DIOR, and NWPU VHR-10, demonstrate the effectiveness and superiority of our proposed method compared with that of other advanced remote-sensing object-detection methods.

Multi-feature fusion: Graph neural network and CNN combining for hyperspectral image classification

Due to its impressive representation power, the graph convolutional network (GCN) has attracted increasing attention in the hyperspectral image (HSI) classification. However, the most of available GCN-based methods for HSI classification utilize superpixels as graph nodes, which ignore the pixel-wise spectral-spatial features. To overcome the issues, we propose a novel multi-feature fusion network (MFGCN), where two different convolutional networks, i.e., multi-scale GCN and multi-scale convolutional neural network (CNN), are utilized in two branches, separately. The multi-scale superpixel-based GCN can reduce the computing power requirements, deal with the problem of labeled deficiency, and refine the multi-scale spatial features from HSI. The multi-scale CNN can extract the multi-scale pixel-wise local features for HSI classification. Furthermore, we introduced a 1D CNN to extract the spectral features for superpixels (nodes), which is different from most existing methods. Finally, a concatenate operation is employed to fuse the complementary multi-scale features. In comparison with the state-of-the-art models on three datasets, the proposed method achieves superior experimental results and outperforms competitive methods.

Sex Differences of Cerebellum and Cerebrum: Evidence from Graph Convolutional Network.

This work aims to exploit a novel graph neural network to predict the sex of the brain topological network, and to find the sex differences in the cerebrum and cerebellum. A two-branch multi-scale graph convolutional network (TMGCN) is designed to analyze the sex differences of the brain. Two complementary templates are used to construct cerebrum and cerebellum networks, respectively, followed by a two-branch sub-network with multi-scale filters and a trainable weighted fusion strategy for the final prediction. Finally, a trainable graph topk-pooling layer is utilized in our model to visualize key brain regions relevant to the prediction. The proposed TMGCN achieves a prediction accuracy of 84.48%. In the cerebellum, the bilateral Crus I-II, lobule VI and VIIb, and the posterior vermis (VI-X) are discriminative for this task. As for the cerebrum, the discriminative brain regions consist of the bilateral inferior temporal gyrus, the bilateral fusiform gyrus, the bilateral parahippocampal gyrus, the bilateral cingulate gyrus, the bilateral medial ventral occipital cortex, the bilateral lateral occipital cortex, the bilateral amygdala, and the bilateral hippocampus. This study tackles the sex prediction problem from a more comprehensive view, and may provide the resting-state fMRI evidence for further study of sex differences in the cerebellum and cerebrum.

An Unsupervised Remote Sensing Change Detection Method Based on Multiscale Graph Convolutional Network and Metric Learning

As a fundamental application, change detection (CD) is widespread in the remote sensing (RS) community. With the increase in the spatial resolution of RS images, high-resolution remote sensing (HRRS) image CD tasks receive growing attention. The change information hidden in multitemporal HRRS images could help discover our planet comprehensively. In the current deep learning era, convolutional neural networks (CNNs) have become one of the most powerful tools for a wide range of RS tasks including HRRS image CD, due to their superb feature learning capacity. However, most of them need a large amount of labeled data to accomplish the CD process, which is challenging or even impractical in many RS applications. Also, given the limited valid receptive field, CNNs can only capture short-range context within HRRS images, which is probably not enough to fully explore change information from the images. To overcome these limitations, in this article, we propose an unsupervised CD method, termed GMCD, based on graph convolutional network (GCN) and metric learning. GMCD consists of a Siamese fully convolution network (FCN), a multiscale dynamic GCN (Mlt-GCN), and a pseudolabel generation mechanism based on metric learning. The Siamese FCN contains a Siamese encoder and a pyramid-shaped decoder, aiming to extract multiscale features and integrate them to generate reliable difference images (DIs). Mlt-GCN focuses on capturing the short- and long-range contextual patterns at feature map level to extract changed and unchanged areas completely. The pseudolabel generation mechanism aims to produce reliable pseudolabels (changed, unchanged, and uncertain) to help accomplish the model training in an unsupervised way. Experiments on four HRRS image CD datasets demonstrate that GMCD outperforms the existing state-of-the-art methods.

Exploit Feature and Relation Hierarchy for Relation Extraction

Existing methods in relation extraction have leveraged the lexical features in the word sequence and the syntactic features in the parse tree. Though effective, the lexical features extracted from the successive word sequence may introduce some noise that has little or no meaningful content. Meanwhile, the syntactic features are usually encoded via graph convolutional networks which have restricted receptive field. In addition, the relation between lexical and syntactic features in the representation space has been largely neglected. To address the above limitations, we propose a multi-scale representation and metric learning framework to exploit the feature and relation hierarchy for RE tasks. Methodologically, we <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">build a lexical and syntactic feature and relation hierarchy</i> in text data. Technically, we first develop <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a multi-scale convolutional neural network</i> to aggregate the non-successive lexical patterns in the word sequence. We also design <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a multi-scale graph convolutional network</i> to increase the receptive field via the coarsened syntactic graph. Moreover, we present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a multi-scale metric learning</i> paradigm to exploit both the feature-level relation between lexical and syntactic features and the sample-level relation between instances with the same or different classes. Extensive experiments on three public datasets for two RE tasks prove that our model achieves a new state-of-the-art performance.

A multiscale graph convolutional network for change detection in homogeneous and heterogeneous remote sensing images

To date, although numerous methods of Change detection (CD) in remote sensing images have been proposed, accurately identifying changes is still a great challenge, due to the difficulties in effectively modeling the features from ground objects with different patterns. In this paper, a novel CD method based on the graph convolutional network (GCN) and multiscale object-based technique is proposed for both homogeneous and heterogeneous images. First, the object-wise high level features are obtained through a pre-trained U-net and the multiscale segmentations. Second, by treating each parcel as a node, the graph representations can be formed and then fed into the proposed multiscale graph convolutional network with each channel corresponding to one scale. The multiscale GCN propagates the label information from a small amount of labeled nodes to the other unlabeled ones. Finally, to comprehensively incorporate the information from the output channels of multiscale GCN, a fusion strategy is designed using the parent–child relationships between scales. Extensive experiments on optical, SAR and heterogeneous optical/SAR data sets demonstrate that the proposed method outperforms some state-of-the-art methods in both qualitative and quantitative evaluations.

Hyperspectral image classification with multi-scale graph convolution network

ABSTRACT Hyperspectral images with high intra-class variability and dimensionality, which greatly weakens the performance of supervised classification methods, especially with the limited number of samples. The graph convolution networks (GCNs) have attracted much attention in the field of hyperspectral classification as it is able to formulate robust spectral-spatial features simultaneously. To this end, we propose a multiscale graph convolutional network based on the spectral-spatial features called MSGCN for hyperspectral image classification. The framework allows us to fuse spectral-spatial features within the graph for node and edge feature generation. Based on that, we further aggregate information from multi-order adjacent nodes to obtain multiscale spectral-spatial features, which proves to be beneficial for hyperspectral image classification. To demonstrate the robustness of the proposed method, three widely used hyperspectral data sets were selected, with the overall accuracy of 99.52, 99.34 and 98.31, respectively.

Facial Expression Recognition With Multiscale Graph Convolutional Networks

Recognizing emotion through facial expression has now been widely applied in our daily lives. Therefore, facial expression recognition (FER) is attracting increasing research interests in the field of artificial intelligence and multimedia. With the development of convolutional neural networks (CNN), end-to-end deep learning frameworks for FER have achieved great success on large-scale datasets. However, these works still face the problems of redundant information and data bias, which obviously decrease the performance of FER. In this article, we propose a novel multiscale graph convolutional network (GCN) based on landmark graphs extracted from facial images. The proposed method is evaluated on different popular datasets. The results show that the proposed method outperforms the traditional deep learning frameworks and achieves more stable performance on different datasets.

Multi-Scale Graph Convolutional Network With Spectral Graph Wavelet Frame

Graph neural networks have achieved impressive progress in solving large and complex graph-structured problems. However, existing methods cannot sufficiently explore the advantages of multi-scale information by enlarging the receptive fields through stacking deep layers or multiplying exponentiated graph Laplacian. They are restricted by the discrete scales rigidly dependent on node-hops or suffer from the over-smoothing issue incurred by long-range low-pass filtering. In this paper, we propose a multi-scale graph convolutional network based on the spectral graph wavelet frame to improve multi-scale representation learning. The proposed network flexibly leverages multi-scale neighboring information with continuous scaling to enhance the discrimination ability of the learned multi-scale representations and offer stable feature extraction ensured by frame bounds in theory. The multi-scale spectral convolutional layer is constructed with a low-pass filter and a sequence of dilated band-pass filters to achieve well-established localization in both vertex and frequency domains. The graph convolutional filters can be realized based on spectral graph wavelet kernel functions analytically formulated using parameterized band-limited filters or adaptively learned from training data. Furthermore, fast approximations of exact spectral filtering are developed for memory efficient operations on sparse tensors with guaranteed prediction accuracy and numerical stability. The proposed network is shown to achieve state-of-the-art performance in extensive experiments on citation networks, bioinformatics graphs, and social networks.