Spatial Graph Convolutional Network Research Articles

Spatio-temporal multi-graph transformer network for joint prediction of multiple vessel trajectories

The vessel trajectory prediction plays a vital role in guaranteeing traffic safety for unmanned surface vehicles and autonomous surface vessels. By leveraging advanced satellite communication technology, AIS provides massive vessel trajectories, significantly enhancing maritime safety and decision-making. This research proposes a spatio-temporal multi-graph transformer network (ST-MGT), aiming to predict multiple vessel trajectories simultaneously. This innovative model amalgamates the capabilities of graph convolutional networks (GCNs) and transformer models to proficiently address the spatial and temporal interactions amongst vessels. The ST-MGT is comprised of three crucial layers. The temporal transformer layer employs sophisticated temporal transformer and memory mechanisms to discern the intricate temporal correlations between vessel movements. The spatial multi-graph transformer layer constructs a comprehensive multi-graph representation to illuminate spatial correlations between vessels. It incorporates a spatial graph convolutional network and transformer to meticulously understand and interpret the diverse and complex spatial interactions amongst varying vessels. Lastly, the ξ-Regularized LSTM (RegLSTM) layer is implemented for predicting vessel trajectories accurately, based on the unraveled spatio-temporal patterns. Extensive and meticulous experiments reveal that our proposed ST-MGT method transcends other state-of-the-art prediction models in robustness and accuracy. The model’s capability to facilitate multi-vessel and multi-step prediction showcases its immense potential and adaptability in intricate and multifaceted navigation environments, underscoring its practical applicability and significance in enhancing maritime navigational safety.

Traffic Speed Prediction Based on Time Classification in Combination With Spatial Graph Convolutional Network

With the advancement of automatic driving and smart city, it is critical to predict traffic information for traffic management, traffic planning, and traffic safety. When predicting traffic information, the spatial structure of the roads will also affect the traffic flow information, such as speed, occupancy rate, etc. The common method either merely focusing on the temporal feature without the considering the spatial structure, or the method of spatial feature extraction is only applicable to Euclidean structure, which does not apply to Non-Euclidean structure. This paper proposes a traffic speed prediction method based on time classification in combination with spatial Graph Convolutional Network. This method employs Gated Recurrent Unit to extract the temporal correlation and Graph Convolutional Network to extract the traffic network’s spatial structure. In consideration of the varying features of traffic speed on weekdays and weekends in the time dimension, time is divided into two types: weekdays and weekends. Since the structure of the road network will not change in the short term in actual process, the same network structure of spatial graph convolution can reasonably be shared in the spatial dimension after which the two sections are fused for training and prediction. Finally, this proposed method is compared to some baseline models to prove the performance. Generally speaking, this strategy produces more accurate prediction results on the PEMS_BAY and METR_LA data sets than the baseline models.

A Self-Trained Spatial Graph Convolutional Network for Unsupervised Human-Related Anomalous Event Detection in Complex Scenes

A Self-Trained Spatial Graph Convolutional Network for Unsupervised Human-Related Anomalous Event Detection in Complex Scenes

LCBP-STGCN: A local cube binary pattern spatial temporal graph convolutional network for micro-expression recognition

Automated micro-expression recognition has become a research highlight in the emotion recognition field. Recent works proposed an LCBP (Local Cube Binary Pattern) method for micro-expression recognition and made full use of spatiotemporal features to represent micro-expressions. Nevertheless, LCBP misses the features while ignoring the underlying discriminative information. In this paper, we present an LCBP-STGCN (Local Cube Binary Pattern Spatial-Temporal Graph Convolutional Network) to resolve the problems of LCBP. A new STGCN with the ability to handle non-Euclidean structure data is proposed to extract high-level features of the micro-expression. STGCN is composed of Spatial Graph Convolutional Network (SGCN) to obtain spatial information and Temporal Convolutional Network (TCN) to capture temporal information of micro-expression. To validly establish the spatiotemporal graph structure of SGCN, we apply ROI (Region of Interest) as node position, LCBP features as node information. By the alternating convolution of SGCN and TCN, high-level spatiotemporal features can be obtained. The extensive experiments on four spontaneous micro-expression datasets of SMIC, CASME I, CASME II, and SAMM demonstrate the proposed LCBP-STGCN can effectively recognize micro-expressions and achieve better performance than some state-of-the-arts.

A Strong and Robust Skeleton-Based Gait Recognition Method with Gait Periodicity Priors

Gait recognition aims to identify people by their walking patterns. The normal human walking is a periodic movement. However, the existing gait recognition methods rarely make use of the gait periodicity. In this paper, we propose the gait Periodicity-inspired Temporal feature Pyramid aggregator (PTP) that introduces gait periodicity priors into the gait feature extraction, resulting in a strong and robust skeleton-based gait recognition method called CycleGait. Specially, inspired by gait periodicity, PTP adopts multiple parallel temporal convolution operators with pyramid temporal kernel sizes to extract gait temporal features. Then PTP cooperates with spatial Graph Convolutional Network (GCN) to form the GCN-PTP network. CycleGait uses this network to extract spatio-temporal gait features from the sequence of skeleton coordinates. Besides, to make CycleGait more robust and have better performance, we feed more gait samples with various gait cycles into CycleGait by the plug-and-play Irregular Pace Converter (IPC) that can automatically convert normal pace into irregular and reasonable pace. Extensive experiments conducted on CASIA-B dataset and OG RGB+D dataset show that CycleGait has excellent performance in various complex scenarios, namely, cross-view and cross-walking-condition, and becomes one of the best SOTA methods, which not only outperforms the existing best gait recognition methods by a large margin, but also exhibits a significant level of robustness.

EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM With Attention Mechanism.

The dynamic uncertain relationship among each brain region is a necessary factor that limits EEG-based emotion recognition. It is a thought-provoking problem to availably employ time-varying spatial and temporal characteristics from multi-channel electroencephalogram (EEG) signals. Although deep learning has made remarkable achievements in emotion recognition, the biological topological information among brain regions does not fully exploit, which is vital for EEG-based emotion recognition. In response to this problem, we design a hybrid model called ST-GCLSTM, which comprises a spatial-graph convolutional network (SGCN) module and an attention-enhanced bi-directional Long Short-Term Memory (LSTM) module. The main advantage of ST-GCLSTM is that it can consider the biological topology information of each brain region to extract representative spatial-temporal features from multiple EEG channels. Specifically, we construct two layers SGCN by introducing adjacency matrices to adaptively learn the intrinsic connection among different EEG channels. Moreover, an attention-enhanced mechanism is placed into a bi-directional LSTM module to extract the crucial spatial-temporal features from sequential EEG data, and then these features serve as the input layer of the classifier to learn discriminative emotion-related features. Extensive experiments on the DEAP, SEED, and SEED-IV datasets demonstrate the effectiveness of the proposed ST-GCLSTM model, revealing that our model had an absolute performance improvement over state-of-the-art strategies.

Adaptive Cross-Attention-Driven Spatial–Spectral Graph Convolutional Network for Hyperspectral Image Classification

Recently, graph convolutional networks (GCNs) have been developed to explore the spatial relationship between pixels, achieving better classification performance of hyperspectral images (HSIs). However, these methods fail to sufficiently leverage the relationship between spectral bands in HSI data. As such, we propose an adaptive cross-attention-driven spatial–spectral graph convolutional network (ACSS-GCN), which is composed of a spatial GCN (Sa-GCN) subnetwork, a spectral GCN (Se-GCN) subnetwork, and a graph cross-attention fusion module (GCAFM). Specifically, Sa-GCN and Se-GCN are proposed to extract the spatial and spectral features by modeling the correlations between spatial pixels and between spectral bands, respectively. Then, by integrating attention mechanism into information aggregation of the graph, the GCAFM, including three parts, i.e., the spatial graph attention block, the spectral graph attention block, and the fusion block, is designed to fuse the spatial and spectral features, and suppress noise interference in Sa-GCN and Se-GCN. Moreover, the idea of the adaptive graph is introduced to explore an optimal graph through backpropagation during the training process. Experiments on two HSI datasets show that the proposed method achieves better performance than other classification methods.

Hybrid Spatiotemporal Graph Convolutional Network for Detecting Landscape Pattern Evolution From Long-Term Remote Sensing Images

The remote sensing time-series change detection algorithm based on the pixel or single landscape patch ignores the change analysis of spatial structure information. Inspired by graph convolutional network (GCN) modeling, a set of landscapes (nodes) and their relationships (edges) are proposed. In this study, a parallel strategy in spatial GCN and progressive strategy in temporal GCN, called the hybrid GCN model network as a holistic framework, was proposed to accurately capture both spatial and temporal variations in landscape patterns based on yearly Landsat time series. A super-patch (i.e., fixed patch class surrounded by a one-hop neighbor patch) was selected as the input of the GCN network. First, a spatial GCN model with three parallel graph convolutional layers was adopted to classify landscape pattern types. Three dominant categories of landscape patterns over the past three decades have been identified. Second, four landscape metrics in super-patches were proposed for the quantitative characterization of changes in landscape patterns. Finally, a temporal GCN model with two progressive graph convolutional layers was used to detect six types of patch changes, which were applied to continuously detect the landscape pattern evolution processes. Regardless of the spatial GCN and temporal GCN, they provided satisfactory performance using the training and validation sets with overall accuracy > 92% and Kappa coefficient > 0.90, and loss values converging to 0.049 and 0.128, respectively, based on NLLLoss function until 500 epochs. It is believed that the hybrid GCN model has great potential for mining possible implicit spatiotemporal relationships and future evolution of landscape patterns.

HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation

An efficient solution to the large-scale recommender system is to represent users and items as binary hash codes in the Hamming space. Towards this end, existing methods tend to code users by modeling their Hamming similarities with the items they historically interact with, which are termed as the first-order similarities in this work. Despite their efficiency, these methods suffer from the suboptimal representative capacity, since they forgo the correlation established by connecting multiple first-order similarities, i.e., the relation among the indirect instances, which could be defined as the high-order similarity. To tackle this drawback, we propose to model both the first- and the high-order similarities in the Hamming space through the user-item bipartite graph. Therefore, we develop a novel learning to hash framework, namely Hamming Spatial Graph Convolutional Networks (HS-GCN), which explicitly models the Hamming similarity and embeds it into the codes of users and items. Extensive experiments on three public benchmark datasets demonstrate that our proposed model significantly outperforms several state-of-the-art hashing models, and obtains performance comparable with the real-valued recommendation models.

Image super-resolution via channel attention and spatial graph convolutional network

Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and obtained remarkable performance. However, most of the existing CNN-based SR methods mainly focus on wider or deeper architecture design, neglecting to discover the latent relationship of features, hence limiting the representational ability of networks. To address this issue, we propose a channel attention and spatial graph convolutional network (CASGCN) for more powerful feature obtaining and feature correlations modeling. The CASGCN is formed by several channel attention and spatial graph (CASG) blocks that incorporate global spatial and channel inter-dependencies for rendering features of each pixel. Inside the CASG block, channel branch and spatial branch are first arranged in a paralleled way, and then are concatenated to effectively learn the representation of each image pixel. Specifically, we use attention mechanism to extract informative features in channel branch while the spatial-aware graph is used in spatial branch to model the global self-similar information. Furthermore, the adjacency matrix in spatial-aware graph is dynamically generated via the Gram matrix to model global correlations between pixels and is shared across the whole network without auxiliary parameters. Extensive experiments on SISR with different degradation models show the effectiveness of our CASGCN in terms of quantitative and visual results.

Spectral–Spatial Graph Convolutional Networks for Semisupervised Hyperspectral Image Classification

Collecting labeled samples is quite costly and time-consuming for hyperspectral image (HSI) classification task. Semisupervised learning framework, which combines the intrinsic information of labeled and unlabeled samples, can alleviate the deficient labeled samples and increase the accuracy of HSI classification. In this letter, we propose a novel semisupervised learning framework that is based on spectral-spatial graph convolutional networks (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> GCNs). It explicitly utilizes the adjacency nodes in graph to approximate the convolution. In the process of approximate convolution on graph, the proposed method makes full use of the spatial information of the current pixel. The experimental results on three real-life HSI data sets, i.e., Botswana Hyperion, Kennedy Space Center, and Indian Pines, show that the proposed S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> GCN can significantly improve the classification accuracy. For instance, the overall accuracy on Indian data is increased from 66.8% (GCN) to 91.6%.