Spatiotemporal Dependence Research Articles

Intricate Spatiotemporal Dependency Learning for Temporal Knowledge Graph Reasoning

Knowledge Graph (KG) reasoning has been an interesting topic in recent decades. Most current researches focus on predicting the missing facts for incomplete KG. Nevertheless, Temporal KG (TKG) reasoning, which is to forecast future facts, still faces with a dilemma due to the complex interactions between entities over time. This article proposes a novel intricate Spatiotemporal Dependency learning Network (STDN) based on Graph Convolutional Network (GCN) to capture the underlying correlations of an entity at different timestamps. Specifically, we first learn an adaptive adjacency matrix to depict the direct dependencies from the temporally adjacent facts of an entity, obtaining its previous context embedding. Then, a Spatiotemporal feature Encoding GCN (STE-GCN) is proposed to capture the latent spatiotemporal dependencies of the entity, getting the spatiotemporal embedding. Finally, a time gate unit is used to integrate the previous context embedding and the spatiotemporal embedding at the current timestamp to update the entity evolutional embedding for predicting future facts. STDN could generate the more expressive embeddings for capturing the intricate spatiotemporal dependencies in TKG. Extensive experiments on WIKI, ICEWS14, and ICEWS18 datasets prove our STDN has the advantage over state-of-the-art baselines for the temporal reasoning task.

Multi-scale feature enhanced spatio-temporal learning for traffic flow forecasting

Traffic flow forecasting is a critical task within Intelligent Transportation Systems (ITS). Its main challenge lies in effectively modeling the complex traffic related big data, including intricate intra-channel and inter-channel correlations, as well as dynamic spatio-temporal dependencies. Furthermore, current methods continue to encounter bottlenecks in extracting and learning complex and dynamic spatio-temporal features from the original traffic data for long-term prediction, resulting in challenges related to model robustness and generalization. In response to these, we introduce a novel deep learning model with multi-scale feature enhancement for traffic flow forecasting, which is based on the attention mechanism and the graph convolution learning framework. We first introduce the integration design of the spatio-temporal dependency features enhancement module with the base attention learning block through a memory embedding layer. Then we propose a traffic network topology features enhancement module with the spatial attention layer, enabling dynamic enhanced learning of spatio-temporal dependency features. This comprehensive approach enables the model to effectively learn complex and dynamic spatio-temporal dependencies, capturing key patterns in traffic flow data. Through extensive experimental evaluations using traffic flow forecasting benchmarks, we have validated the superior performance of the proposed model over the state-of-the-art.

TrajectoFormer: Transformer-Based Trajectory Prediction of Autonomous Vehicles with Spatio-temporal Neighborhood Considerations

Accurate trajectory prediction of autonomous vehicles is crucial for ensuring road safety. Predicting precise and accurate trajectories is still considered a challenging problem because of the intricate spatio-temporal dependencies among the vehicles. Our study primarily focuses on resolving this issue by introducing a comprehensive system called “TrajectoFormer”, which can effectively represent the spatio-temporal dependency between vehicles. In this system, we have conducted preprocessing on the NGSIM dataset by constructing an 8-neighborhood for each vehicle that represents the spatio-temporal dependency between vehicles effectively. Second, we have deployed a transformer network that captures dependencies between the target vehicle and its neighbor from the constructed neighborhood and predicts future trajectories for the target vehicle with notably reduced training times and significant accuracy compared to existing methods. Experiments on both NGSIM US-101 and US-I80 show that our proposed approach outperforms the other benchmarks in terms of showing low RMSE value for the 5-s prediction horizon of trajectory prediction. Our conducted ablation study also underscores the effectiveness of each component of our proposed TrajectoFormer model relative to traditional time-series prediction models.

Spatiotemporal gated traffic trajectory simulation with semantic-aware graph learning

Traffic trajectories of various vehicles, bicycles and pedestrians can help understand the traffic dynamics in a fine-grained manner like traffic flow, traffic congestion and ride-hailing demand. The comprehensive usage of traffic trajectory data has not been fully investigated due to the prevalent privacy concerns and commercial limitations. The traffic trajectory simulation task has emerged to generate high-fidelity trajectories in demand for downstream tasks to fill the gap between the scarce trajectory data and the widespread applications. Previous state-of-the-art methods build the spatiotemporal dependencies of trajectories with Graph Neural Networks (GNNs) under generative adversarial training, yielding better yet unstable trajectory quality. We observe that the unsatisfied synthetic trajectories are caused by the insufficient spatiotemporal modeling of road networks and trajectory semantics. In this paper, we propose a novel SpatioTEmporal GAted (STEGA) framework equipped with semantic-aware graph learning for traffic trajectory simulation to enable the explicit modeling of spatiotemporal dependencies throughout the learning pipeline. On the one hand, STEGA employs a graph encoder with the semantics of road networks for the spatial points of a trajectory, together with a time encoder for the time points. On the other hand, STEGA devises two spatiotemporal gates with the semantic graphs for the predictions of the future trajectory. Boosted by the semantic-aware graph learning, the proposed STEGA outperforms the state-of-the-art counterparts consistently at both macro- and micro-level metrics on two datasets. Elaborate ablation studies and downstream tasks of the synthetic trajectories further demonstrate the superiority of STEGA. Our code is available at https://github.com/Star607/STEGA.

Micro-Doppler based Human Activity Recognition using ABOA based Dual Spatial Convolution with Gated Recurrent Unit

The through-wall capability, device-free detection of radar-based human activity recognition are drawing a lot of interest from both academics and industry. The majority of radar-based systems do not yet combine signal analysis and feature extraction in the frequency domain and the time domain. Applications like smart homes, assisted living, and monitoring rely on human identification and activity recognition (HIAR). Radar has a number of advantages over other sensing modalities, such as the ability to shield users' privacy and conduct contactless sensing. The article introduces a new human tracking system that uses radar and a classifier called Dual Spatial Convolution Gated Recurrent Unit (DSC-GRU) to identify the subject and their behavior. The system follows the person and identifies the type of motion whenever it detects movement. One important feature is the integration of the GRU with the DSC unit, which allows the model to simultaneously capture the spatiotemporal dependence. Present prediction models just take into account spatial features that are immediately adjacent to each other, disregarding or just superimposing global spatial features when taking spatial correlation into account. A new dependency graph is created by calculating the correlation among nodes using the correlation coefficient; this graph represents the global spatial dependence, while the classic static graph represents the neighboring spatial dependence in the DSC unit. The DSC unit goes a step further by using a modified gated mechanism to quantify the various contributions of both local and global spatial correlation. While previous models performed worse, the suggested model outperformed them with an accuracy of 99.45 percent and a precision of 97.15 percent.

Chasing the heat: Unraveling urban hyperlocal air temperature mapping with mobile sensing and machine learning

Many cities face unprecedented high temperatures with increasing extreme events. Heatwaves pose significant health risks, including cardiovascular diseases, heatstroke, and dehydration. Mapping urban near-surface air temperature (Tair) is crucial for understanding thermal exposure and addressing climate change. Previous studies relied on satellite-derived land surface temperature (LST) and stationary monitoring, but high spaio-temporal Tair mapping is still a challenge. This study optimized a mobile sensing scheme using an electric bicycle platform with environmental and image sensors, and deep learning captured local-scale urban factors. A spatio-temporal data fusion model that consisted of three parts, temporal trend extraction, locality analysis, and neighborhood effect analysis, generated hyperlocal Tair maps. The Results from Beijing demonstrated the effectiveness of the framework, achieving the lowest MAE of 0.02 °C. Optimized data collection and the new model achieved accurate temperature predictions and thermal exposure assessment. Efficiency enhanced sensing strategy was also proposed. The study highlights local-scale factors and spatio-temporal dependencies in addressing heatwaves and climate change impacts in urban areas.

Advancing urban traffic accident forecasting through sparse spatio-temporal dynamic learning

Traffic accidents have emerged as one of the most public health safety matters, raising concerns from both the public and urban administrators. The ability to accurately predict traffic accident not only supports the governmental decision-making in advance but also enhances public confidence in safety measures. However, the efficacy of traditional spatio-temporal prediction models are compromised by the skewed distributions and sparse labeling of accident data. To this end, we propose a Sparse Spatio-Temporal Dynamic Hypergraph Learning (SST-DHL) framework that captures higher-order dependencies in sparse traffic accidents by combining hypergraph learning and self-supervised learning. The SST-DHL model incorporates a multi-view spatiotemporal convolution block to capture local correlations and semantics of traffic accidents, a cross-regional dynamic hypergraph learning model to identify global spatiotemporal dependencies, and a two-supervised self-learning paradigm to capture both local and global spatiotemporal patterns. Through experimentation on New York City and London accident datasets, we demonstrate that our proposed SST-DHL exhibits significant improvements compared to optimal baseline models at different sparsity levels. Additionally, it offers enhanced interpretability of results by elucidating complex spatio-temporal dependencies among various traffic accident instances. Our study demonstrates the effectiveness of the SST-DHL framework in accurately predicting traffic accidents, thereby enhancing public safety and trust.

Is Single Enough? A Joint Spatiotemporal Feature Learning Framework for Multivariate Time Series Prediction.

A fuzzy cognitive map (FCM) is a simple but effective tool for modeling and predicting time series. This article focuses on the problem of multivariate time series prediction (TSP), which is essential and challenging in data mining. Although several FCM-based approaches have been designed to solve this problem, their feature extraction module designed for single mode falls short in capturing the nonlinear spatiotemporal dependencies among variates, thereby resulting in low prediction accuracy in forecasting multivariate time series, which shows that the single mode learning is not enough. Therefore, in this article, we propose a joint spatiotemporal feature learning framework for multivariate TSP, where a mix-resolution spatial module consisting of multiple sparse autoencoders (SAEs) is designed to extract the feature series with different spatial resolutions, and a mix-order spatiotemporal module concluding multiple high-order FCMs (HFCMs) is designed to model the spatiotemporal dynamics of these feature series. Finally, the outputs of the two modules are concatenated to predict future values. We refer to this framework as the spatiotemporal FCM (STFCM). Especially, an efficient learning algorithm is designed to update the integral weights of STFCM based on the batch gradient descent algorithm when it deems necessary. We validate the performance of the STFCM on four real-world datasets. Compared with the existing state-of-the-art (SOTA) methods, the experimental results not only show the advantages of the two designed modules in the STFCM but also show the excellent performance of the STFCM.

Enhancing traffic flow prediction: Dual-branch graph convolutional network with rough data inference and adaptive spatial dependencies

Traffic flow prediction is a significant application of deep learning in spatio-temporal forecasting analysis. Existing research faces challenges that hinder prediction accuracy. One challenge is the inadequate capturing of spatial dependencies in traffic flow due to fixed pre-defined graph structures. Moreover, manually designed prior graphs still have limitations in extracting spatial features. Another challenge is the instability in short-term prediction performance when pre-defined graphs are completely abandoned in favor of parameter training. Additionally, ordinary RNN sequence convolution methods also struggle to capture long-term sequential patterns in large historical traffic data, leading to gradient vanishing or exploding issues. To address these challenges, we proposes a graph convolutional network for traffic flow prediction. We combine an improved prior graph with an adaptive graph to form a dual-branch spatio-temporal neural network. In the first branch, we introduce a time graph based on rough data inference to complement the predefined static graph. In the second branch, we construct an adaptive learning framework that dynamically learns the adjacency matrix and captures global road information. By utilizing enhanced multi-scale gated convolution, we extract spatio-temporal dependencies. Our method surpasses most baseline approaches according to extensive experiments conducted on public datasets.

A spatio-temporal graph neural network for fall prediction with inertial sensors

Falls are the leading cause of unintentional human injury, having become a public health event of strong social concern. The fall prediction technology based on wearable inertial sensors is a relatively reliable solution in human activity monitoring, a user scenario with mobility and high information privacy sensitivity, and has the advantages of low cost, small size, and high precision. However, a key challenge in sensor-based fall prediction is the fact that a fall event can often occur in various configurations of fall poses together with their own spatio-temporal dependencies. This leads us to define a spatio-temporal model to explicitly characterize these internal configurations of poses. In particular, we introduce a graph neural network with spatio-temporal topological structure to encode such latent relations among poses by capturing representative patterns in fall events. Moreover, a human body orientation estimator is devised to represent human low limbs information and as a result, separate pose dependencies are globally consistent. Empirical evaluations on two benchmark datasets and one in-house dataset suggest our approach significantly outperforms the state-of-the-art methods.

Spatio-Temporal Pivotal Graph Neural Networks for Traffic Flow Forecasting

Traffic flow forecasting is a classical spatio-temporal data mining problem with many real-world applications. Recently, various methods based on Graph Neural Networks (GNN) have been proposed for the problem and achieved impressive prediction performance. However, we argue that the majority of existing methods disregarding the importance of certain nodes (referred to as pivotal nodes) that naturally exhibit extensive connections with multiple other nodes. Predicting on pivotal nodes poses a challenge due to their complex spatio-temporal dependencies compared to other nodes. In this paper, we propose a novel GNN-based method called Spatio-Temporal Pivotal Graph Neural Networks (STPGNN) to address the above limitation. We introduce a pivotal node identification module for identifying pivotal nodes. We propose a novel pivotal graph convolution module, enabling precise capture of spatio-temporal dependencies centered around pivotal nodes. Moreover, we propose a parallel framework capable of extracting spatio-temporal traffic features on both pivotal and non-pivotal nodes. Experiments on seven real-world traffic datasets verify our proposed method's effectiveness and efficiency compared to state-of-the-art baselines.

Spatio-Temporal Fusion for Human Action Recognition via Joint Trajectory Graph

Graph Convolutional Networks (GCNs) and Transformers have been widely applied to skeleton-based human action recognition, with each offering unique advantages in capturing spatial relationships and long-range dependencies. However, for most GCN methods, the construction of topological structures relies solely on the spatial information of human joints, limiting their ability to directly capture richer spatio-temporal dependencies. Additionally, the self-attention modules of many Transformer methods lack topological structure information, restricting the robustness and generalization of the models. To address these issues, we propose a Joint Trajectory Graph (JTG) that integrates spatio-temporal information into a uniform graph structure. We also present a Joint Trajectory GraphFormer (JT-GraphFormer), which directly captures the spatio-temporal relationships among all joint trajectories for human action recognition. To better integrate topological information into spatio-temporal relationships, we introduce a Spatio-Temporal Dijkstra Attention (STDA) mechanism to calculate relationship scores for all the joints in JTG. Furthermore, we incorporate the Koopman operator into the classification stage to enhance the model's representation ability and classification performance. Experiments demonstrate that JT-GraphFormer achieves outstanding performance in human action recognition tasks, outperforming state-of-the-art methods on the NTU RGB+D, NTU RGB+D 120, and N-UCLA datasets.

Estimating the mixed layer depth of the global ocean by combining multisource remote sensing and spatiotemporal deep learning

ABSTRACT Estimating the ocean mixed layer depth (MLD) is crucial for studying the atmosphere-ocean interaction and global climate change. Satellite observations can accurately estimate the MLD over large scales, effectively overcoming the limitation of sparse in situ observations and reducing uncertainty caused by estimation based on in situ and reanalysis data. However, combining multisource satellite observations to accurately estimate the global MLD is still extremely challenging. This study proposed a novel Residual Convolutional Gate Recurrent Unit (ResConvGRU) neural networks, to accurately estimate global MLD along with multisource remote sensing data and Argo gridded data. With the inherent spatiotemporal nonlinearity and dependence of the ocean dynamic process, the proposed method is effective in spatiotemporal feature learning by considering temporal dependence and capturing more spatial features of the ocean observation data. The performance metrics show that the proposed ResConvGRU outperforms other well-used machine learning models, with a global determination coefficient (R2) and a global root mean squared error (RMSE) of 0.886 and 17.83 m, respectively. Overall, the new deep learning approach proposed is more robust and advantageous in data-driven spatiotemporal modeling for retrieving ocean MLD at the global scale, and significantly improves the estimation accuracy of MLD from remote sensing observations.

Spatio-Temporal Self-Attention Network for Origin–Destination Matrix Prediction in Urban Rail Transit

Short-term origin–destination (OD) prediction in urban rail transit (URT) is vital for improving URT operation. However, due to the problems such as the unavailability of the OD matrix of the current day, high dimension and long-range spatio-temporal dependencies, it is difficult to further improve the prediction accuracy of an OD matrix. In this paper, a novel spatio-temporal self-attention network (SSNet) for OD matrix prediction in URT is proposed to further improve the prediction accuracy. In the proposed SSNet, a lightweight yet effective spatio-temporal self-attention module (STSM) is proposed to capture complex long-range spatio-temporal dependencies, thus helping improve the prediction accuracy of the proposed SSNet. Additionally, the finished OD matrices on previous days are used as the only data source without the passenger flow data on the current day in the proposed SSNet, which makes it possible to predict the OD matrices of all time intervals on the current day before the operation of the current day. It is demonstrated by experiments that the proposed SSNet outperforms three advanced deep learning methods for short-term OD prediction in URT, and the proposed STSM plays an important role in improving the prediction accuracy.

VST-PCA: A Land Use Change Simulation Model Based on Spatiotemporal Feature Extraction and Pre-Allocation Strategy

Land use/cover change (LUCC) refers to the phenomenon of changes in the Earth’s surface over time. Accurate prediction of LUCC is crucial for guiding policy formulation and resource management, contributing to the sustainable use of land, and maintaining the health of the Earth’s ecosystems. LUCC is a dynamic geographical process involving complex spatiotemporal dependencies. Existing LUCC simulation models suffer from insufficient spatiotemporal feature learning, and traditional cellular automaton (CA) models exhibit limitations in neighborhood effects. This study proposes a cellular automaton model based on spatiotemporal feature learning and hotspot area pre-allocation (VST-PCA). The model utilizes the video swin transformer to acquire transformation rules, enabling a more accurate capture of the spatiotemporal dependencies inherent in LUCC. Simultaneously, a pre-allocation strategy is introduced in the CA simulation to address the local constraints of neighborhood effects, thereby enhancing the simulation accuracy. Using the Chongqing metropolitan area as the study area, two traditional CA models and two deep learning-based CA models were constructed to validate the performance of the VST-PCA model. Results indicated that the proposed VST-PCA model achieved Kappa and FOM values of 0.8654 and 0.4534, respectively. Compared to other models, Kappa increased by 0.0322–0.1036, and FOM increased by 0.0513–0.1649. This study provides an accurate and effective method for LUCC simulation, offering valuable insights for future research and land management planning.

PAR-Net: An Enhanced Dual-Stream CNN-ESN Architecture for Human Physical Activity Recognition.

Physical exercise affects many facets of life, including mental health, social interaction, physical fitness, and illness prevention, among many others. Therefore, several AI-driven techniques have been developed in the literature to recognize human physical activities. However, these techniques fail to adequately learn the temporal and spatial features of the data patterns. Additionally, these techniques are unable to fully comprehend complex activity patterns over different periods, emphasizing the need for enhanced architectures to further increase accuracy by learning spatiotemporal dependencies in the data individually. Therefore, in this work, we develop an attention-enhanced dual-stream network (PAR-Net) for physical activity recognition with the ability to extract both spatial and temporal features simultaneously. The PAR-Net integrates convolutional neural networks (CNNs) and echo state networks (ESNs), followed by a self-attention mechanism for optimal feature selection. The dual-stream feature extraction mechanism enables the PAR-Net to learn spatiotemporal dependencies from actual data. Furthermore, the incorporation of a self-attention mechanism makes a substantial contribution by facilitating targeted attention on significant features, hence enhancing the identification of nuanced activity patterns. The PAR-Net was evaluated on two benchmark physical activity recognition datasets and achieved higher performance by surpassing the baselines comparatively. Additionally, a thorough ablation study was conducted to determine the best optimal model for human physical activity recognition.

Accommodating spatio-temporal dependency in airline demand modeling

The objective of the current study is to examine monthly air passenger departures at the airport level considering spatial interactions between airports. In this study, we develop a novel spatial grouped generalized ordered probit (SGGOP) model system of monthly air passenger departures at the airport level. Specifically, we estimate two variants of spatial models including spatial lag model and spatial error model. In the presence of repeated demand measures for the airports, we also consider temporal variations of spatial correlation effects among proximally located airports by employing space and time-based weight matrix. The proposed model is estimated using monthly air passenger departures for five years for 369 airports across the US. The proposed spatial model is implemented using composite marginal likelihood (CML) approach that offers a computationally feasible framework. From the estimation results, it is evident that air passenger departures at the airport level are influenced by different factors including MSA specific demographic characteristics, built environment characteristics, airport specific factors, spatial factors, and temporal factors. Moreover, spatial autocorrelation parameter is found to be significant validating our hypothesis of the presence of common unobserved factors associated with the spatial unit of analysis. In this study, we also perform a validation analysis to examine the predictive performance of the proposed spatial models. The results highlight the superiority of spatial error model compared to spatial lag model and the independent model that ignores the spatial interactions. Finally, we undertake an elasticity analysis to quantify the impact of the independent variables.

Towards a comprehensive characterization of spatio-temporal dependence of light-induced electromagnetic forces in dielectric liquids

The interaction of localized light with matter generates optical electrostriction within dielectric fluids, leading to a discernible change in the refractive index of the medium according to the excitation’s light profile. This optical force holds critical significance in optical manipulation and plays a fundamental role in numerous photonic applications. In this study, we demonstrate the applicability of the pump-probe, photo-induced lensing (PIL) method to investigate optical electrostriction in various dielectric liquids. Notably, the thermal and nonlinear effects are observed to be temporally decoupled from the electrostriction effects, facilitating isolated observation of the latter. Our findings provide a comprehensive explanation of optical forces in the context of the recently introduced microscopic Ampère electromagnetic formalism, which is grounded in the dipolar approximation of electromagnetic sources within matter and characterizes electrostriction as an electromagnetic-induced stress within the medium. Here, the optical force density is re-obtained through a new Lagrangian approach.

Representation modeling learning with multi-domain decoupling for unsupervised skeleton-based action recognition

Skeleton-based action recognition is one of the basic researches in computer vision. In recent years, the unsupervised contrastive learning paradigm has achieved great success in skeleton-based action recognition. However, previous work often treated input skeleton sequences as a whole when performing comparisons, lacking fine-grained representation contrast learning. Therefore, we propose a contrastive learning method for Representation Modeling with Multi-domain Decoupling (RMMD), which extracts the most significant representations from input skeleton sequences in the temporal domain, spatial domain and frequency domain, respectively. Specifically, in the temporal and spatial domains, we propose a multi-level spatiotemporal mining reconstruction module (STMR) that iteratively reconstructs the original input skeleton sequences to highlight spatiotemporal representations under different actions. At the same time, we introduce position encoding and a global adaptive attention matrix, balancing both global and local information, and effectively modeling the spatiotemporal dependencies between joints. In the frequency domain, we use the discrete cosine transform (DCT) to achieve temporal-frequency conversion, discard part of the interference information, and use the frequency self-attention (FSA) and multi-level aggregation perceptron (MLAP) to deeply explore the frequency domain representation. The fusion of the temporal domain, spatial domain and frequency domain representations makes our model more discriminative in representing different actions. Besides, we verify the effectiveness of the model on the NTU RGB+D and PKU-MMD datasets. Extensive experiments show that our method outperforms existing unsupervised methods and achieves significant performance improvements in downstream tasks such as action recognition and action retrieval.

Multi-scale persistent spatiotemporal transformer for long-term urban traffic flow prediction

Long-term urban traffic flow prediction is an important task in the field of intelligent transportation, as it can help optimize traffic management and improve travel efficiency. To improve prediction accuracy, a crucial issue is how to model spatiotemporal dependency in urban traffic data. In recent years, many studies have adopted spatiotemporal neural networks to extract key information from traffic data. However, most models ignore the semantic spatial similarity between long-distance areas when mining spatial dependency. They also ignore the impact of predicted time steps on the next unpredicted time step for making long-term predictions. Moreover, these models lack a comprehensive data embedding process to represent complex spatiotemporal dependency. This paper proposes a multi-scale persistent spatiotemporal transformer (MSPSTT) model to perform accurate long-term traffic flow prediction in cities. MSPSTT adopts an encoder-decoder structure and incorporates temporal, periodic, and spatial features to fully embed urban traffic data to address these issues. The model consists of a spatiotemporal encoder and a spatiotemporal decoder, which rely on temporal, geospatial, and semantic space multi-head attention modules to dynamically extract temporal, geospatial, and semantic characteristics. The spatiotemporal decoder combines the context information provided by the encoder, integrates the predicted time step information, and is iteratively updated to learn the correlation between different time steps in the broader time range to improve the model's accuracy for long-term prediction. Experiments on four public transportation datasets demonstrate that MSPSTT outperforms the existing models by up to 9.5% on three common metrics.