Spatiotemporal Attention Mechanism Research Articles

Predicting air quality using a multi-scale spatiotemporal graph attention network

As urbanization accelerates, air quality has become a pressing concern. Accurate air quality prediction is essential for informed governmental decision-making and for protecting public health. Variations in air quality are influenced by complex multi-scale spatiotemporal processes. Existing research primarily relies on capturing single spatiotemporal features of air quality to predict changes. Meanwhile, when constructing spatiotemporal dynamic graphs, the inherent characteristics of the input data and the comprehensive effects of both global and local influences are not fully considered. To address these problems, we propose a graph-attention-based approach, named Multi-scale Spatiotemporal Graph Attention Network (MSTGAN). MSTGAN addresses the intricate spatiotemporal patterns of air quality across various scales through three key components: (1) a multistation transformer to model the temporal patterns of air quality at individual monitoring stations; (2) a bilinear spatiotemporal attention mechanism to capture the spatiotemporal dynamic global dependencies among all stations in a region; and (3) a set of spatiotemporal dependence graph-coupled Chebyshev graph convolution gate recurrent units to extract and aggregate the local spatiotemporal features of interrelated stations. Experiments conducted on three real-world datasets demonstrated that MSTGAN achieved significant improvements of 4.2%, 3.9%, and 7.8% in the mean absolute error, root mean square error, and R2 evaluation metrics, respectively, compared to seven state-of-the-art time-series forecasting methods. This code is publicly available at https://github.com/HPSCIL/MSTGAN-airquality-prediction.

Spatiotemporal Mechanism-Based Spacetimeformer Network for InSAR Deformation Prediction and Identification of Retrogressive Thaw Slumps in the Chumar River Basin

The increasing incidence of retrogressive thaw slumps (RTSs) in permafrost regions underscores the need for detailed spatial and temporal analysis using InSAR technology to monitor and predict dynamic changes in the process of RTSs. Nevertheless, current InSAR deformation forecasting methods employing deep learning strategies such as the traditional long short-term memory (LSTM) and recent transformer models encounter difficulties in effectively capturing temporal features. Moreover, they are limited in their ability to directly integrate spatial information. In this paper, an innovative deep learning approach named Spacetimeformer is proposed for predicting medium- and short-term InSAR deformation of RTSs in the Chumar River area. This method employs a transformer architecture with a spatiotemporal attention mechanism, which enhances the long-term prediction capabilities of time series models and dynamic spatial modeling. It is applicable to multivariate InSAR spatiotemporal deformation prediction problems. The findings include a list of 72 RTSs compiled based on derived InSAR deformation maps and Sentinel-2 optical images, of which 64 have an average deformation rate exceeding 10 mm/year, indicating signs of permafrost degradation. The density distribution of the displacement maps predicted by the Spacetimeformer model aligned well with the InSAR deformation maps obtained from the small baseline subset (SBAS) method, with the overall prediction deviation controlled within 20 mm. In addition, the point-scale prediction results were compared with LSTM and transformer models. This study indicates that the Spacetimeformer network achieved good results in predicting the deformation of RTSs, with a root mean square error of 1.249 mm. The Spacetimeformer method for deformation prediction with the spacetime mechanism presented in this study can serve as a general framework for multivariate deformation prediction based on InSAR results. It can also quantitatively assess the spatial deformation characteristics and deformation trends of RTSs.

Traffic Flow Prediction with Random Walks on Graph and Spatiotemporal Bidirectional Attention Transformer

Traffic flow prediction is crucial in intelligent transportation systems. Considering the severe disruptions caused by traffic accidents or congestion, a time series model is developed for traffic flow prediction based on multiple random walks on graphs (MRWG) and the bidirectional spatiotemporal attention mechanism (BSAM), which can adapt to both normal and exceptional situations. The MRWG mechanism is applied to capture spatial features of urban areas during traffic accidents and congestion, especially the spatial dependencies among neighboring regions. Further, a local position attention module is applied to acquire the spatial correlations between different regions to investigate their impact on the global area, while a local temporal attention module is adopted to extract short-term periodic time correlations from traffic flow data. Finally, a spatiotemporal bidirectional attention module is applied to simultaneously extract both the temporal and spatial correlations of the historical traffic flow data in order to generate the output prediction. Experiments have been conducted on NYCTaxi and NYCBike datasets with abnormal events, and the results indicate that the developed model can efficiently predict traffic flow in abnormal events, especially short-term traffic disruptions, outperforming the baseline methods under both abnormal and normal conditions.

Predicting Vessel Trajectories Using ASTGCN with StemGNN-Derived Correlation Matrix

This study proposes a vessel position prediction method using attention spatiotemporal graph convolutional networks, which addresses the issue of low prediction accuracy due to less consideration of inter-feature dependencies in current vessel trajectory prediction methods. First, the method cleans the vessel trajectory data and uses the Time-ratio trajectory compression algorithm to compress the trajectory data, avoiding data redundancy and providing feature points for vessel trajectories. Second, the Spectral Temporal Graph Neural Network (StemGNN) extracts the correlation matrix that describes the relationship between multiple variables as a priori matrix input to the prediction model. Then the vessel trajectory prediction model is constructed, and the attention mechanism is added to the spatial and temporal dimensions of the trajectory data based on the spatio-temporal graph convolutional network at the same time as the above operations are performed on different time scales. Finally, the features extracted from different time scales are fused through the full connectivity layer to predict the future trajectories. Experimental results show that this method achieves higher accuracy and more stable prediction results in trajectory prediction. The attention-based spatio-temporal graph convolutional networks effectively capture the spatio-temporal correlations of the main features in vessel trajectories, and the spatio-temporal attention mechanism and graph convolution have certain interpretability for the prediction results.

Short-term photovoltaic energy generation for solar powered high efficiency irrigation systems using LSTM with Spatio-temporal attention mechanism

Solar irrigation systems should become more practical and efficient as technology advances. Automation and AI-based technologies can optimize solar energy use for irrigation while reducing environmental impacts and costs. These innovations have the potential to make agriculture more environmentally friendly and sustainable. Solar irrigation system implementation can be hampered by a lack of technical expertise in installation, operation, and maintenance. It must be technically and economically feasible to be practical and continuous. Due to weather and solar irradiation, photovoltaic power generation is difficult for high-efficiency irrigation systems. As a result, more precise photovoltaic output calculations could improve solar power systems. Customers should benefit from increased power plant versatility and high-quality electricity. As a result, an artificial intelligence-powered automated irrigation power-generation system may improve the existing efficiency. To predict high-efficiency irrigation system power outputs, this study proposed a spatial and temporal attention block-based long-short-term memory (LSTM) model. Using MSE, RMSE, and MAE, the results have been compared to pre-existing ML and a simple LSTM network. Moreover, it has been found that our model outperformed cutting-edge methods. MAPE was improved by 6–7% by increasing Look Back (LB) and Look Forward (LF). Future goals include adapting the technology for wind power production and improving the proposed model to harness customer behavior to improve forecasting accuracy.

Short-term load forecasting of regional integrated energy system based on spatio-temporal convolutional graph neural network

Accurate short-term load forecasting has significant reference value for the preliminary planning and subsequent coordinated control of regional integrated energy systems. However, main limitations are: (1) Insufficient exploration of the complex correlations among diverse loads and between loads and meteorological factors; and (2) Feature information extraction only from the time dimension, neglecting the spatial dimension, resulting in incomplete feature information extraction. This paper proposes a new method for short-term load forecasting, namely the Attention Spatial–temporal Graph Convolutional Network (ATSTGCN) method. Firstly, the Grey Relational Analysis (GRA) method is used to analyze the correlation between diverse loads, meteorological features, and load features, constructing the input data for the graph neural network. Then, synchronous graph convolution is introduced to extract information between features. Next, the spatio-temporal attention mechanism is incorporated into the synchronous graph convolution to capture spatio-temporal features and reduce information loss using a spatio-temporal window. Finally, a short-term load forecasting method based on ATSTGCN for regional integrated energy systems is constructed. Extensive simulation experiments are conducted on the publicly available dataset from Arizona State University’s online system. The experimental results show that compared to other methods, the proposed method improves MAPE, MAE, and RMSE by 5.6%, 11.7%, and 11.6%.

Correlation-based feature selection and parallel spatiotemporal networks for efficient passenger flow forecasting in metro systems

This paper presents a novel framework for predicting metro passenger flow that is both interpretable and computationally efficient. The proposed method first uses a correlation-based spatiotemporal feature selection strategy (Cor-STFS) to identify the optimal input scheme for the prediction model, effectively reducing unnecessary interference. The framework then introduces a new multivariate passenger flow prediction architecture called STA-PTCN-BiGRU, which combines a spatiotemporal attention (STA) mechanism, parallel temporal convolutional networks (PTCN), and bidirectional gated recurrent units (BiGRU) to capture the dynamic internal patterns of passenger flow. By utilising parallel computing, this architecture significantly reduces resource consumption. The effectiveness of the proposed approach is evaluated using four datasets from the Shanghai Metro. Experimental results show that the new method outperforms baseline approaches in terms of root mean square error (RMSE), mean absolute error (MAE), and symmetric mean absolute percentage error (SMAPE), achieving average reductions of 9.98%, 8.08%, and 13.29% in these metrics, respectively.

Short-term forecasting of surface solar incident radiation on edge intelligence based on AttUNet

Solar energy has emerged as a key industry in the field of renewable energy due to its universality, harmlessness, and sustainability. Accurate prediction of solar radiation is crucial for optimizing the economic benefits of photovoltaic power plants. In this paper, we propose a novel spatiotemporal attention mechanism model based on an encoder-translator-decoder architecture. Our model is built upon a temporal AttUNet network and incorporates an auxiliary attention branch to enhance the extraction of spatiotemporal correlation information from input images. And utilize the powerful ability of edge intelligence to process meteorological data and solar radiation parameters in real-time, adjust the prediction model in real-time, thereby improving the real-time performance of prediction. The dataset utilized in this study is sourced from the total surface solar incident radiation (SSI) product provided by the geostationary meteorological satellite FY4A. After experiments, the SSIM has been improved to 0.86. Compared with other existing models, our model has obvious advantages and has great prospects for short-term prediction of surface solar incident radiation.

Image Super-Resolution Reconstruction Model Based on Multi-Feature Fusion

Due to the limitations of imaging equipment and image transmission conditions on daily image acquisition, the images acquired are usually low-resolution images, and it will cost a lot of time and economic costs to increase image resolution by upgrading hardware equipment. In this paper, we propose an image super-resolution reconstruction algorithm based on spatio-temporal-dependent residual network MSRN, which fuses multiple features. The algorithm uses the surface feature extraction module to extract the input features of the image, and then uses the deep residual aggregation module to adaptively learn the deep features, and then fuses multiple features and learns the global residual. Finally, the high-resolution image is obtained through the up-sampling module and the reconstruction module. In the model structure, different convolution kernels and jump connections are used to extract more high-frequency information, and spatio-temporal attention mechanism is introduced to focus on more image details. The experimental results show that compared with SRGAN, VDSR and Laplacian Pyramid SRN, the proposed algorithm finally achieves better reconstruction effect, and the image texture details are clearer under different scaling factors. In objective evaluation, the peak signal-to-noise ratio (PSNR) and structure similarity (SSIM) of the proposed algorithm are improved compared with SRGAN.

Enhancing Major Depressive Disorder Diagnosis with Dynamic-Static Fusion Graph Neural Networks.

Major Depressive Disorder (MDD) is a debilitating, complex mental condition with unclear mechanisms hindering diagnostic progress. Research links MDD to abnormal brain connectivity using functional magnetic resonance imaging (fMRI). Yet, existing fMRI-based MDD models suffer from limitations, including neglecting dynamic network traits, lacking interpretability, and struggling with small datasets. We present DSFGNN, a novel graph neural network framework addressing these issues for improved MDD diagnosis. DSFGNN employs a graph isomorphism encoder to model static and dynamic brain networks, achieving effective fusion of temporal and spatial information through a spatiotemporal attention mechanism, thereby enhancing interpretability. Furthermore, we incorporate a causal disentangling module and orthogonal regularization module to augment the model's expressiveness. We evaluate DSFGNN on the Rest-meta-MDD dataset, yielding superior results compared to the best baseline. Besides, extensive ablation studies and interpretability analysis confirm DSFGNN's effectiveness and potential for biomarker discovery.

Adaptive Spatial–Temporal Aware Graph Learning for EEG-Based Emotion Recognition

An intelligent emotion recognition system based on electroencephalography (EEG) signals shows considerable potential in various domains such as healthcare, entertainment, and education, thanks to its portability, high temporal resolution, and real-time capabilities. However, the existing research in this field faces limitations stemming from the nonstationary nature and individual variability of EEG signals. In this study, we present a novel EEG emotion recognition model, named GraphEmotionNet, designed to enhance the accuracy of EEG-based emotion recognition through the incorporation of a spatiotemporal attention mechanism and transfer learning. The proposed GraphEmotionNet model can effectively learn the intrinsic connections between EEG channels and construct an adaptive graph. This graph’s adaptive nature is crucial in optimizing spatial–temporal graph convolutions, which in turn enhances spatial–temporal feature characterization and contributes to the process of emotion classification. Moreover, an integration of domain adaptation aligns the extracted features across different domains, further alleviating the impact of individual EEG variability. We evaluate the model performance on two benchmark databases, employing two types of cross-validation protocols: within-subject cross-validation and cross-subject cross-validation. The experimental results affirm the model’s efficacy in extracting EEG features linked to emotional semantics and demonstrate its promising performance in emotion recognition.

Spatio-Temporal-Attention-Based Vehicle Trajectory Prediction Considering Multi-Vehicle Interaction in Mixed Traffic Flow

As a link connecting the environmental perception system and the decision-making system, accurate obstacle trajectory prediction provides a reliable guarantee of correct decision-making by autonomous vehicles. Oriented toward a mixed human-driven and machine-driven traffic environment, a vehicle trajectory prediction algorithm based on an encoding–decoding framework composed of a multiple-attention mechanism is proposed. Firstly, a directed graph is used to describe vehicle–vehicle motion dependencies. Then, by calculating the repulsive force between vehicles using a priori edge information based on the artificial potential field theory, vehicle–vehicle interaction coefficients are extracted via a graph attention mechanism (GAT). Subsequently, after concatenating the vehicle–vehicle interaction feature with the encoded vehicle trajectory vectors, a spatio-temporal attention mechanism is applied to determine the coupling relationship of hidden vectors. Finally, the predicted trajectory is generated by a gated recurrent unit (GRU) decoder. The training and evaluation of the proposed model were conducted on the NGSIM public dataset. The test results demonstrated that compared with existing baseline models, our approach has fewer prediction errors and better robustness. In addition, introducing artificial potential fields into the attention mechanism causes the model to have better interpretability.

Local motion feature extraction and spatiotemporal attention mechanism for action recognition

Local motion feature extraction and spatiotemporal attention mechanism for action recognition

Method for Predicting Transverse Wave Velocity Using a Gated Recurrent Unit Based on Spatiotemporal Attention Mechanism

Abstract Transverse wave velocity plays an important role in seismic exploration and reservoir assessment in the oil and gas industry. Due to the lack of transverse wave velocity data from actual production activities, it is necessary to predict transverse wave velocity based on longitudinal wave velocity and other reservoir parameters. This paper proposes a fusion network based on spatiotemporal attention mechanism and gated recurrent unit (STAGRU) due to the significant correlation between the transverse wave velocity and reservoir parameters in the spatiotemporal domain. In the case of tight sandstone reservoirs in the Junggar Basin, the intersection plot technique is used to select four well logging parameters that are sensitive to transverse wave velocity: longitudinal wave velocity, density, natural gamma, and neutron porosity. The autocorrelation technique is employed to analyze the depth-related correlation of well logging curves. The relationship between the spatiotemporal characteristics of these well logging data and the network attention weights is also examined to validate the rationale behind incorporating the spatiotemporal attention mechanism. Finally, the actual measurement data from multiple wells are utilized to analyze the performance of the training set and test set separately. The results indicate that the predictive accuracy and generalization ability of the proposed STAGRU method are superior to the single-parameter fitting method, multiparameter fitting method, Xu-White model method, GRU network, and 2DCNN-GRU hybrid network. This demonstrates the feasibility of the transverse wave velocity prediction method based on the spatiotemporal attention mechanism in the study of rock physics modeling for tight sandstone reservoirs.

GSAA: A Novel Graph Spatiotemporal Attention Algorithm for Smart City Traffic Prediction

With the development of 5G and Internet of Things technologies, the application process of smart transportation in smart cities continues to advance. Sensors are a key source of information for smart transportation, and their data commonly includes complicated traffic scene information. Urban traffic scheduling and efficiency can be significantly increased by deploying data from smart sensors to forecast traffic flows. Despite the fact that some related works have focused on the prediction task of traffic flows, they have not completely mined the traffic spatiotemporal information present in smart sensor data. We offer a novel graph spatio-temporal attention algorithm (GSAA) for traffic prediction in this paper. To completely exploit the geographical and temporal correlations among complicated roadways for traffic forecast, the algorithm combines a spatiotemporal attention mechanism with a graph neural network.To take full advantage of how much effect various hyperparameters provide, deep reinforcement learning is used to obtain the optimal hyperparameters while the predictive model is trained. Experimental results on real-world public datasets show that the algorithm proposed in this paper achieves performance improvements of about 5.47% and 13.10% over the MAE (mean absolute error) than the best baseline strategies for short-term and long-term traffic forecasting, respectively.

Multi-head spatio-temporal attention based parallel GRU architecture: a novel multi-sensor fusion method for mechanical fault diagnosis

The sensor signals with multiple measuring points and data types not only bring sufficient information, but also put forward more stringent requirements for multi-sensor fusion efficiency and fault feature extraction capability. The redundancy and conflicts in the information of multi-sensor signals often hinder the accurate extraction of crucial fault features. To address this problem, our study proposes an intelligent mechanical fault diagnosis method, which is based on a multi-head spatio-temporal attention mechanism and parallel gated recurrent units (GRUs) architecture. This method utilizes multiple attention heads to model the correlation information in spatial and temporal dimensions, and employs a parallel GRU network for targeted feature extraction. Finally, it combines local features from different attention heads to achieve flexible scheduling of various spatio-temporal attention modes. This novel application and fusion approach of multi-head attention enables accurate identification of the spatio-temporal value differences in the collected multi-sensor signals from multiple perspectives. Experimental results on multiple mechanical fault datasets show that the proposed method performs well in multi-sensor signals based mechanical fault diagnosis tasks and can maintain effectiveness under small samples and imbalanced data conditions.

Traffic Flow Forecasting of Graph Convolutional Network Based on Spatio-Temporal Attention Mechanism

Traffic Flow Forecasting of Graph Convolutional Network Based on Spatio-Temporal Attention Mechanism

Traj2Traj: A road network constrained spatiotemporal interpolation model for traffic trajectory restoration

AbstractIn transportation, the trajectory data generated by various mobile vehicles equipped with GPS modules are essential for traffic information mining. However, collecting trajectory data is susceptible to various factors, resulting in the lack and even error of the data. Missing trajectory data could not correctly reflect the actual situation and also affect the subsequent research work related to the trajectory. Although increasing efforts are paid to restore missing trajectory data, it still faces many challenges: (1) the difficulty of data restoration because traffic trajectories are unstructured spatiotemporal data and show complex patterns; and (2) the difficulty of improving trajectory restoration efficiency because traditional trajectory interpolation is computationally arduous. To address these issues, a novel road network constrained spatiotemporal interpolation model, namely Traj2Traj, is proposed in this work to restore the missing traffic trajectory data. The model is constructed with a seq2seq network and integrates a potential factor module to extend environmental factors. Significantly, the model uses a spatiotemporal attention mechanism with the road network constraint to mine the latent information in time and space dimensions from massive trajectory data. The Traj2Traj model completes the road‐level restoration according to the entire trajectory information. We present the first attempt to omit the map‐matching task when the trajectory is restored to solve the time‐consuming problem of map matching. Extensive experiments conducted on the provincial vehicle GPS data sets from April 2018 to June 2018 provided by the Fujian Provincial Department of Transportation show that the Traj2Traj model outperforms the state‐of‐the‐art models.

Pseudo-Label-Vector-Guided Parallel Attention Network for Remaining Useful Life Prediction

Prognostic health management (PHM) has become important in many industries as a critical technology to increase machine stability and operational efficiency. Recently, various methods using deep learning to estimate the remaining useful life (RUL) as a core task of PHM have been proposed. However, the existing attention methods do not explicitly capture the correlation between temporal and spatial time series, reducing the RUL prediction accuracy. This article proposes a novel RUL prediction algorithm using a spatiotemporal attention mechanism based on the pseudo-label vectors to solve this problem. The proposed attention network uses the pseudo-label vector learned in the intermediate prediction process as a query vector to focus on time sequence data related to the RUL. Therefore, compared with conventional attention models that extract correlations for all the sequences, the proposed model captures features directly related to RUL with less computational cost. Experiments have been performed on two widely used datasets, and the experimental results show that the proposed approach outperforms the state of the art for root-mean-square error, with averages 4.27 and 3039 in the NASA Commercial Modular Aero-Propulsion System Simulation dataset and the IEEE PHM 2012 Prognostic challenge dataset, respectively. In addition, the analysis in the experiment reveals that the proposed model has better interpretability than the existing models by obtaining the correlation between time-series data and the RUL through the attention score in terms of time and features.

Sustainable groundwater management using stacked LSTM with deep neural network

Groundwater is a vital water resource and plays a major role in human life, production, irrigation, and development of the country based on economically. Due to irregular rainfall and drought in the summer season, storage of groundwater is essential for the usage of multipurpose. The prediction of groundwater using the spatiotemporal attention mechanism is the main goal of this research. With the rapid growth of urbanization, population, and industrialization, the resource of groundwater has become vulnerable to depletion. Therefore, it is necessary for groundwater resource management in the aspects of quality and quantity. The groundwater and its demand are indirectly proportional to each other. This imbalance criterion brings more problems in groundwater availability. Effective and efficient planning is required to face this dilemma. In facing the groundwater challenge many research works have been implemented. The issues are inefficient and fail to predict the demand for water requirements. To overcome these issues this paper proposed managing groundwater by using stacked LSTM with a deep neural network (SLSTM-DNN) for performing demand prediction in the collected dataset. An accuracy rate of 81.32% was obtained for CNN, 82.34% obtained for DNN, 88.12% obtained for LSTM, and the proposed accuracy rate achieves 91.45%.