Spatial-temporal Correlation Research Articles

STViT: Improving Self-Supervised Multi-Camera Depth Estimation with Spatial-Temporal Context and Adversarial Geometry Regularization (Student Abstract)

Multi-camera depth estimation has recently garnered significant attention due to its substantial practical implications in the realm of autonomous driving. In this paper, we delve into the task of self-supervised multi-camera depth estimation and propose an innovative framework, STViT, featuring several noteworthy enhancements: 1) we propose a Spatial-Temporal Transformer to comprehensively exploit both local connectivity and the global context of image features, meanwhile learning enriched spatial-temporal cross-view correlations to recover 3D geometry. 2) to alleviate the severe effect of adverse conditions, e.g., rainy weather and nighttime driving, we introduce a GAN-based Adversarial Geometry Regularization Module (AGR) to further constrain the depth estimation with unpaired normal-condition depth maps and prevent the model from being incorrectly trained. Experiments on challenging autonomous driving datasets Nuscenes and DDAD show that our method achieves state-of-the-art performance.

Disentanglement-Guided Spatial-Temporal Graph Neural Network for Metro Flow Forecasting (Student Abstract)

In recent intelligent transportation applications, metro flow forecasting has received much attention from researchers. Most prior arts endeavor to explore spatial or temporal dependencies while ignoring the key characteristic patterns underlying historical flows, e.g., trend and periodicity. Although the multiple granularity distillations or spatial dependency correlation can promote the flow estimation. However, the potential noise and spatial dynamics are under-explored. To this end, we propose a novel Disentanglement-Guided Spatial-Temporal Graph Neural Network or DGST to address the above concerns. It contains a Disentanglement Pre-training procedure for characteristic pattern disentanglement learning, a Characteristic Pattern Prediction for different future characteristic explorations, and a Spatial-Temporal Correlation for spatial-temporal dynamic learning. Experiments on a real-world dataset demonstrate the superiority of our DGST.

Spectral-temporal convolutional approach for PV systems output power forecasting: Case studies in single-site and multi-site

Accurate predictions of photovoltaic (PV) and wind power outputs are indispensable for integrating additional renewable energy sources into the grid. Photovoltaic energy is influenced by weather conditions and cloud movement, constituting the primary factors contributing to stochastic characteristics and uncertainty in predictive accuracy. Consequently, recent studies have focused on developing forecasting methods that utilize weather factors or nearby farms as multi-variable inputs. This study introduces several challenges, and one of the most significant goals is to capture the correlation between data series (spatial feature) and temporal correlation within each series (temporal feature) simultaneously. However, most previous work has concentrated solely on the time or frequency domain. This work presents a novel approach based on the spectral and frequency domain, which efficiently captures spatial–temporal correlations via Graph Fourier Transform (GFT) and Discrete Fourier Transform (DFT). In this manner, spatial correlations are clearly represented via the adjacency matrix of GFT, and temporal correlations are clarified via DFT. The study provides specific evidence concerning the interpretability and extraction efficiency of the adjacency matrix in the context of multi-variable, single-site, and multi-site data. Simultaneously, it demonstrates the effectiveness of the representation approach in the spectral domain. The results pointed out the forecasting method proposed has higher forecasting accuracy than the benchmark model.

Robust change detection for remote sensing images based on temporospatial interactive attention module

Change Detection (CD) is a vital monitoring method in Earth observation, especially pertinent for land-use analysis, city management, and disaster damage assessment. However, in the era of constellation interconnection and air-sky collaboration, the changes in the Regions Of Interest (ROI) cause many false detections due to geometric perspective rotation and temporal style difference. In response to these challenges, we introduce CDNeXt, this framework elucidates a robust and efficient method for combining Siamese networks based on the pre-trained backbone with the innovative Temporospatial Interactive Attention Module (TIAM) for remote sensing imagery. The CDNeXt can be categorized into four primary components: Encoder, Interactor, Decoder, and Detector. Notably, the Interactor, powered by TIAM, queries and rebuilds spatial perspective dependencies and temporal style correlations from binary temporal features extracted by the Encoder to enlarge the difference of ROI change. Culminating the process, the Detector integrates the hierarchical features generated by the Decoder, subsequently producing a binary change mask. We have achieved new State-Of-The-Art (SOTA) performance in change detection, with our method surpassing existing techniques on four benchmark datasets: an F1 score of 82.63% on SYSU-CD, 87.14% on LEVIR-CD+, 66.71% on S2Looking, and 71.11% BANDON. To further validate the effectiveness of the TIAM, we compared it to other attention modules in both interactive and non-interactive modes. Our code is available on GitHub: https://github.com/wjj282439449/CDNeXt.

DeepMeshCity: A Deep Learning Model for Urban Grid Prediction

Urban grid prediction can be applied to many classic spatial-temporal prediction tasks such as air quality prediction, crowd density prediction, and traffic flow prediction, which is of great importance to smart city building. In light of its practical values, many methods have been developed for it and have achieved promising results. Despite their successes, two main challenges remain open: a) how to well capture the global dependencies and b) how to effectively model the multi-scale spatial-temporal correlations? To address these two challenges, we propose a novel method— DeepMeshCity , with a carefully-designed Self-Attention Citywide Grid Learner ( SA-CGL ) block comprising of a self-attention unit and a Citywide Grid Learner ( CGL ) unit. The self-attention block aims to capture the global spatial dependencies, and the CGL unit is responsible for learning the spatial-temporal correlations. In particular, a multi-scale memory unit is proposed to traverse all stacked SA-CGL blocks along a zigzag path to capture the multi-scale spatial-temporal correlations. In addition, we propose to initialize the single-scale memory units and the multi-scale memory units by using the corresponding ones in the previous fragment stack, so as to speed up the model training. We evaluate the performance of our proposed model by comparing with several state-of-the-art methods on four real-world datasets for two urban grid prediction applications. The experimental results verify the superiority of DeepMeshCity over the existing ones. The code is available at https://github.com/ILoveStudying/DeepMeshCity.

Attention Mechanism with Spatial-Temporal Joint Deep Learning Model for the Forecasting of Short-Term Passenger Flow Distribution at the Railway Station

Accurate understanding of passenger flow distribution is crucial for effective station crowd management. However, due to the complexity and randomness of passenger flow and the unclear spatial-temporal correlation between functional areas within the station, predicting the spatiotemporal distribution dynamics of inflow and future short-term distribution trends is challenging. Emerging deep learning models offer valuable insights for accurately predicting passenger flow distribution. Thus, we propose a deep learning architecture, named “ST-Bi-LSTM,” which combines a bidirectional long short-term memory network with a spatial-temporal attention mechanism. Initially, we outline the methodologies of Bi-LSTM, the DeepWalk-based spatial attention mechanism, and the temporal attention mechanism. The spatial attention mechanism is employed to extract station spatial network topology information and enhance the representation of passenger flow characteristics in highly correlated areas during the forecasting process. Simultaneously, the temporal attention Bi-LSTM is utilized for capturing temporal correlations. The architecture comprises four branches dedicated to station real-time video monitoring data, spatial network topology, function area attributes, and train timetables. Subsequently, leveraging in-station CCTV data, passenger travel behavior data, and train timetables, we apply the architecture to the Tianjin West High-Speed Railway Station. We conduct a comparative analysis of the prediction performance and time complexity of the proposed architecture against existing baseline models, demonstrating superior performance and robustness exhibited by the ST-Bi-LSTM model (achieving a reduction in RMSE of over 10%). This study facilitates the transition of station management from passive response to active prediction of station passenger flow dynamics.

Mdpg: a novel multi-disease diagnosis prediction method based on patient knowledge graphs.

Diagnosis prediction, a key factor in enhancing healthcare efficiency, remains a focal point in clinical decision support research. However, the time-series, sparse and multi-noise characteristics of electronic health record (EHR) data make it a great challenge. Existing methods commonly address these issues using RNNs and incorporating medical prior knowledge from medical knowledge bases, but they neglect the local spatial characteristics and spatial-temporal correlation of the data. Consequently, we propose MDPG, a diagnosis prediction model based on patient knowledge graphs. Initially, we represent the electronic visit records of patients as a patient-centered temporal knowledge graph, capturing the local spatial structure and temporal characteristics of the visit information. Subsequently, we design the spatial graph convolution block, temporal self-attention block, and spatial-temporal synchronous graph convolution block to capture the spatial, temporal, and spatial-temporal correlations embedded in them, respectively. Ultimately, we accomplish the prediction of patients' future states through multi-label classification. We conduct comprehensive experiments on two real-world datasets independently and evaluate the results using visit-level precision@k and code-level accuracy@k metrics. The experimental results demonstrate that MDPG outperforms all baseline models, yielding the best performance.

MVSTT: A Multiview Spatial-Temporal Transformer Network for Traffic-Flow Forecasting.

Accurate traffic-flow prediction remains a critical challenge due to complicated spatial dependencies, temporal factors, and unpredictable events. Most existing approaches focus on single-or dual-view learning and thus face limitations in systematically learning complex spatial-temporal features. In this work, we propose a novel multiview spatial-temporal transformer (MVSTT) network that can effectively learn complex spatial-temporal domain correlations and potential patterns from multiple views. First, we examine a temporal view and design a short-range gated convolution component from a short-term subview, and a long-range gated convolution component from a long-term subview. These two components effectively aggregate knowledge of the temporal domain at multiple granularities and mine patterns of node evolution across time steps. Meanwhile, in the spatial view, we design a dual-graph spatial learning module that captures fixed and dynamic spatial dependencies of nodes, as well as the evolution patterns of edges, from the static and dynamic graph subviews, respectively. In addition, we further design a spatial-temporal transformer to mine different levels of spatial-temporal features through multiview knowledge fusion. Extensive experiments on four real-world traffic datasets show that our method consistently outperforms the state-of-the-art baseline. The code of MVSTT is available at https://github.com/JianSoL/MVSTT.

A Cross-Modal Generative Adversarial Network for Scenarios Generation of Renewable Energy

Scenarios data of renewable energy resources plays an essential role in the study of mitigating the risk in the power system due to their intermittent nature. Existing researches have mainly focused on how to generate high-quality time series based scenarios with corrupted or missing observed data. However, the multimodality of renewable energy resources is largely ignored from the perspective of the spatial-temporal correlation which can significantly strengthen the renewable energy prediction and optimization. To make full use of these multi-modal data and further improve the data quality of scenarios, a cross-modal Generative Adversarial Network (cGAN) model is proposed with the setting of two spatial-temporal transformer models. In cGAN, the task of scenarios generation is formulated as a probability approximation problem. Then, a cross-modal scenarios generation framework is constructed to process data from multi-modal observations. Theoretically, it can generate infinite number of uncertain scenarios data via the repeated random noise sampling technique. Extensive experiments are carried out at a NREL photovoltaic power dataset and a real-world wind power dataset, respectively. The results validate the state-of-the-art performance of cGAN, even though the validation dataset is missing at random. Moreover, the results also show that the setting of spatial-temporal transformer clearly explains the contributions of different modal data, and stabilize the training process of cGAN. Finally, a stochastic day-head economic dispatch is also studied to show the practical value of cGAN.

Spatial-Temporal Attention TCN-Based Link Prediction for Opportunistic Network

Link prediction for opportunistic networks faces the challenges of frequent changes in topology and complex and variable spatial-temporal information. Most existing studies focus on temporal or spatial features, ignoring ample potential information. In order to better capture the spatial-temporal correlations in the evolution of networks and explore their potential information, a link prediction method based on spatial-temporal attention and temporal convolution network (STA-TCN) is proposed. It slices opportunistic networks into discrete network snapshots. A state matrix based on topology information and attribute information is constructed to represent snapshots. Time convolutional networks and spatial-temporal attention mechanisms are employed to learn spatial-temporal information. Furthermore, to better improve link prediction performance, the proposed method converts the auto-correlation error into non-correlation error. On three real opportunistic network datasets, ITC, MIT, and Infocom06, experimental results demonstrate the superior predictive performance of the proposed method compared to baseline models, as shown by improved AUC and F1-score metrics.

Day-Ahead Demand Response Potential Forecasting Model Considering Dynamic Spatial-Temporal Correlation Based on Directed Graph Structure

Day-Ahead Demand Response Potential Forecasting Model Considering Dynamic Spatial-Temporal Correlation Based on Directed Graph Structure

FAST-CA: Fusion-based Adaptive Spatial–Temporal Learning with Coupled Attention for airport network delay propagation prediction

The issue of delay propagation prediction in airport networks has garnered increasing global attention, particularly due to its profound impact on operational efficiency and passenger satisfaction in modern air transportation systems. Despite research advancements in this domain, existing methodologies often fall short of comprehensively addressing the challenges associated with predicting delay propagation in airport networks, especially in terms of handling complex spatial–temporal dependencies and sequence couplings. In response to the complex challenge of predicting delay propagation in airport networks, we introduce the Fusion-based Adaptive Spatial–Temporal Learning with Coupled Attention (FAST-CA) framework. FAST-CA is an innovative model that integrates dynamic and adaptive graph learning, coupled attention mechanisms, periodicity feature extraction, and multifaceted information fusion modules. This holistic approach enables a thorough analysis of the interplay between flight departure and arrival delays and the spatial–temporal correlations within airport networks. Rigorously evaluated on two extensive real-world datasets, our model consistently outperforms current state-of-the-art baseline models, showcasing superior predictive performance and the effective learning capabilities of its intricately designed modules. Our research highlights the criticality of analyzing spatial–temporal relationships and the dynamics of flight coupling, offering significant theoretical and practical contributions to the advancement and management of air transportation systems.

Dynamic multiple-graph spatial-temporal synchronous aggregation framework for traffic prediction in intelligent transportation systems.

Accurate traffic prediction contributes significantly to the success of intelligent transportation systems (ITS), which enables ITS to rationally deploy road resources and enhance the utilization efficiency of road networks. Improvements in prediction performance are evident by utilizing synchronized rather than stepwise components to model spatial-temporal correlations. Some existing studies have designed graph structures containing spatial and temporal attributes to achieve spatial-temporal synchronous learning. However, two challenges remain due to the intricate dynamics: (a) Accounting for the impact of external factors in spatial-temporal synchronous modeling. (b) Multiple perspectives in constructing spatial-temporal synchronous graphs. To address the mentioned limitations, a novel model named dynamic multiple-graph spatial-temporal synchronous aggregation framework (DMSTSAF) for traffic prediction is proposed. Specifically, DMSTSAF utilizes a feature augmentation module (FAM) to adaptively incorporate traffic data with external factors and generate fused features as inputs to subsequent modules. Moreover, DMSTSAF introduces diverse spatial and temporal graphs according to different spatial-temporal relationships. Based on this, two types of spatial-temporal synchronous graphs and the corresponding synchronous aggregation modules are designed to simultaneously extract hidden features from various aspects. Extensive experiments constructed on four real-world datasets indicate that our model improves by 3.68-8.54% compared to the state-of-the-art baseline.

Exploring contribution of phytoplankton cell death to settleable particulate organic carbon in the East China Sea in spring

Phytoplankton's death contributes to marine settleable particulate organic matter (POM). In this study, we used laboratory cultivation of different algal species to identify a positive correlation between the cumulative number of dead algal cells and POC>75 (carbon content of the settleable POM). The contribution coefficient of cell death to POC>75 varied among different algal species. Additionally, the field survey and incubation experiment were conducted in the East China Sea (ECS) to explore the spatial-temporal correlation between phytoplankton death and POC>75. The results concluded that phytoplankton death was the main factor controlling POC>75. In the ECS, the relationship between the surface cumulative mass of POC>75 and the cumulative number of dead cells followed: Cumulative mass of POC>75(mg) = 0.487 × Cumulative number of dead cells (/104) + 0.069. This study provided a methodology to quantitatively explain the relationship between phytoplankton death and settleable POM.

A Novel Spatial–Temporal Deep Learning Method for Metro Flow Prediction Considering External Factors and Periodicity

Predicting metro traffic flow is crucial for efficient urban planning and transit management. It enables cities to optimize resource allocation, reduce congestion, and enhance the overall commuter experience in rapidly urbanizing environments. Nevertheless, metro flow prediction is challenging due to the intricate spatial–temporal relationships inherent in the data and the varying influence of external factors. To model spatial–temporal correlations considering external factors, a novel spatial–temporal deep learning framework is proposed in this study. Firstly, mutual information is utilized to select the highly corrected stations of the examined station. Compared with the traditional correlation calculation methods, mutual information is particularly advantageous for analyzing nonlinear metro flow data. Secondly, metro flow data reflecting the historical trends from different time granularities are incorporated. Additionally, the external factor data that influence the metro flow are also considered. Finally, these multiple sources and dimensions of data are combined and fed into the deep neural network to capture the complex correlations of multi-dimensional data. Sufficient experiments are designed and conducted on the real dataset collected from Xi’an subway to verify the effectiveness of the proposed model. Experimental results are comprehensively analyzed according to the POI information around the subway station.

Effects of Re on blade load temporal-spatial distribution and correlation with flow stability of transonic compressor under inlet distortion

Compressor blade operating conditions and aerodynamic load change periodically owing to inlet distortion. The effects of low Re are often unavoidable when the inlet distortion is encountered in real compressor operation. Blade load distribution and internal flow field are both affected by variations in Re. In this paper, the effects of Re on the blade load distribution patterns under 60° of total pressure distortion is investigated by numerical simulation. The distortion intensity has been set to 0.05, and four cases with Reynolds number index of 1, 0.3, 0.2 and 0.1 are chosen for further analysis. The findings demonstrate that the variation of Re affects the compressor flow field mainly by modifying the aerodynamic profile of the blade, which also results in the change of the temporal and spatial distribution of the aerodynamic load. Furthermore, the aerodynamic blade profile varies significantly by Reynolds number in different spanwise regions. The blade time constant decreases in the mid-span area, while it increases in the tip. This ultimately leads to a delay in the location of the tip maximum load, changing the starting position of the induced stall.

An adaptive adjacency matrix-based graph convolutional recurrent network for air quality prediction

In recent years, air pollution has become increasingly serious and poses a great threat to human health. Timely and accurate air quality prediction is crucial for air pollution early warning and control. Although data-driven air quality prediction methods are promising, there are still challenges in studying spatial–temporal correlations of air pollutants to design effective predictors. To address this issue, a novel model called adaptive adjacency matrix-based graph convolutional recurrent network (AAMGCRN) is proposed in this study. The model inputs Point of Interest (POI) data and meteorological data into a fully connected neural network to learn the weights of the adjacency matrix thereby constructing the self-ringing adjacency matrix and passes the pollutant data with this matrix as input to the Graph Convolutional Network (GCN) unit. Then, the GCN unit is embedded into LSTM units to learn spatio-temporal dependencies. Furthermore, temporal features are extracted using Long Short-Term Memory network (LSTM). Finally, the outputs of these two components are merged and air quality predictions are generated through a hidden layer. To evaluate the performance of the model, we conducted multi-step predictions for the hourly concentration of PM2.5, PM10 and O3 at Fangshan, Tiantan and Dongsi monitoring stations in Beijing. The experimental results show that our method achieves better predicted effects compared with other baseline models based on deep learning. In general, we designed a novel air quality prediction method and effectively addressed the shortcomings of existing studies in learning the spatio-temporal correlations of air pollutants. This method can provide more accurate air quality predictions and is expected to provide support for public health protection and government environmental decision-making.

Abnormal data detection and recovery of sensors network based on spatiotemporal deep learning methodology

This paper proposes a novel deep-learning-enabled framework for abnormal data detection and recovery considering spatial-temporal correlation among sensors. The wavelet scattering transform is adopted to replace the first layer of one-dimension convolutional neural network classification model. The multiple typical sensor failure modes of sensor are identified including bias, drifting, gain, precision degradation, and three complete faults (constant; constant + noise; noise). The bidirectional long-short time memory (BLSTM) network is developed to recover abnormal data through positive and negative propagation. The BLSTM cell involving forget, input, and output gate is utilized to adaptively transmit valuable information of long-term data. Different sensor configurations are conducted to select the optimal input sensors of BLSTM model. The proposed models are then validated by three engineering cases. The results show that the failure mode diagnosis and recovery accuracy are over 90 %. Sensor selection has a large impact on the data reconstruction, and selecting sensors with higher correlation can achieve a higher accuracy. In addition, the data recovery accuracy of the proposed BLSTM model surpasses that of traditional long-short time memory models.

Hierarchical generalization of dual unitarity

Quantum dynamics with local interactions in lattice models display rich physics, but is notoriously hard to study. Dual-unitary circuits allow for exact answers to interesting physical questions in clean or disordered one- and higher-dimensional quantum systems. However, this family of models shows some non-universal features, like vanishing correlations inside the light-cone and instantaneous thermalization of local observables. In this work we propose a generalization of dual-unitary circuits where the exactly calculable spatial-temporal correlation functions display richer behavior, and have non-trivial thermalization of local observables. This is achieved by generalizing the single-gate condition to a hierarchy of multi-gate conditions, where the first level recovers dual-unitary models, and the second level exhibits these new interesting features. We also extend the discussion and provide exact solutions to correlators with few-site observables and discuss higher-orders, including the ones after a quantum quench. In addition, we provide exhaustive parametrizations for qubit cases, and propose a new family of models for local dimensions larger than two, which also provides a new family of dual-unitary models.

An ultra-short-term wind power prediction method based on spatial-temporal attention graph convolutional model

Accurate and robust wind power forecasting (WPF) is great significance to ensure the safe and stable operation of the power system and promote the transformation of low-carbon energy. However, the high randomness and intermittency of wind power bring great challenges when designing reliable forecasting models. In this paper, a novel spatial-temporal attention graph convolutional network model is proposed. Firstly, the spatial attention mechanism is used to aggregate and extract the spatial correlations of the raw wind power data. Secondly, the temporal attention mechanism is applied to capture the temporal correlations. Then, the extracted spatial-temporal correlations were put into the temporal convolution network and the spatial convolution network to further obtain the temporal and spatial dependencies. Finally, the wind power forecasting results is output through the full connection layer. The proposed method is verified by using wind power data from real wind farm in China. The experimental results reveal that the proposed depth spatiotemporal prediction model has more significant advantages than other advanced models in terms of prediction accuracy and stability.