Spatio-temporal Network Research Articles

Power allocation using spatio-temporal graph neural networks and reinforcement learning

Power allocation using spatio-temporal graph neural networks and reinforcement learning

Adaptive Decision Spatio-temporal neural ODE for traffic flow forecasting with Multi-Kernel Temporal Dynamic Dilation Convolution

Traffic flow prediction is crucial for efficient traffic management. It involves predicting vehicle movement patterns to reduce congestion and enhance traffic flow. However, the highly non-linear and complex patterns commonly observed in traffic flow pose significant challenges for this task. Current Graph Neural Network (GNN) models often construct shallow networks, which limits their ability to extract deeper spatio-temporal representations. Neural ordinary differential equations for traffic prediction address over-smoothing but require significant computational resources, leading to inefficiencies, and sometimes deeper networks may lead to poorer predictions for complex traffic information. In this study, we propose an Adaptive Decision spatio-temporal Neural Ordinary Differential Network, which can adaptively determine the number of layers of ODE according to the complexity of traffic information. It can solve the over-smoothing problem better, improving overall efficiency and prediction accuracy. In addition, traditional temporal convolution methods make it difficult to deal with complex and variable traffic time information with a large time span. Therefore, we introduce a multi-kernel temporal dynamic expansive convolution to handle the traffic time information. Multi-kernel temporal dynamic expansive convolution employs a dynamic dilation strategy, dynamically adjusting the network’s receptive field across levels, effectively capturing temporal dependencies, and can better adapt to the changing time data of traffic information. Additionally, multi-kernel temporal dynamic expansive convolution integrates multi-scale convolution kernels, enabling the model to learn features across diverse temporal scales. We evaluated our proposed method on several real-world traffic datasets. Experimental results show that our method outperformed state-of-the-art benchmarks.

Forecasting sea surface temperature during typhoon events in the Bohai Sea using spatiotemporal neural networks

Advanced neural network methods can effectively combine temporal and spatial information, allowing us to exploit complex relationships in the prediction of sea surface temperature (SST). Therefore, the authors have developed an attention-based context fusion network model recently, known as ACFN, to enhance the underlying spatiotemporal correlations in SST data. In a previous preliminary long-term evaluation, ACFN has demonstrated substantial improvements over several state-of-the-art baseline models, such as Convolutional Long-Short Term Memory (ConvLSTM) and Predictive Recurrent Neural Network (PredRNN). However, these methods have not been widely used for predicting SST during typhoons, and their performance has not been thoroughly evaluated. This study focuses on three specific typhoons, In-fa (2021), Lekima (2019), and Rumbia (2018), as case studies. The results show that ACFN consistently outperforms ConvLSTM and PredRNN, as demonstrated during Typhoon In-fa with a mean absolute error of 0.27 °C and a root-mean-square error of 0.33 °C for a 1-day forecast time. This study also examines the impact of forecast time on predictive skill, revealing that the performance of ACFN, as expected, gradually decreases with longer forecast times. This thorough evaluation of the ACFN model provides a valuable reference for using deep learning in predicting regional typhoon SST. Plain language summaryIn previous work, we developed an attention-based context fusion network model. This spatiotemporal neural network model has been successfully applied to short-term sea surface temperature (SST) prediction in the Bohai Sea, utilizing only current and previous 1- to 10-day SST data as input to forecast SST for the subsequent 1- to 10-day period. In this study, we further evaluated the capability of the model to predict SST during typhoons in the Bohai Sea, and the results were very promising. When predicting SST during Typhoon In-fa, the present spatiotemporal neural network model had a mean absolute error of only 0.27 °C for a 1-day forecast time, and a correlation coefficient of 0.74. On the other hand, comparison with baseline spatiotemporal neural network models shows that the present model is more effective in extracting input information. The current model has resolved the constraint that its predictive capability does not diminish as the forecast time increases, thereby enhancing the rationality of the prediction outcomes. In conclusion, the present spatiotemporal neural network model demonstrates its efficacy in forecasting typhoon-induced SST in the Bohai Sea, offering valuable insights for applications in meteorology and oceanography.

Integrating Spatio-Temporal Graph Convolutional Networks with Convolutional Neural Networks for Predicting Short-Term Traffic Speed in Urban Road Networks

The rapid expansion of large urban areas underscores the critical importance of road infrastructure. An accurate understanding of traffic flow on road networks is essential for enhancing civil services and reducing fuel consumption. However, traffic flow is influenced by a complex array of factors and perpetually changing conditions, making comprehensive prediction of road network behavior challenging. Recent research has leveraged deep learning techniques to identify and forecast traffic flow and road network conditions, enhancing prediction accuracy by extracting key features from diverse factors. In this study, we performed short-term traffic speed predictions for road networks using data from Mobileye sensors mounted on taxis in Daegu City, Republic of Korea. These sensors capture the road network flow environment and the driver’s intentions. Utilizing these data, we integrated convolutional neural networks (CNNs) with spatio-temporal graph convolutional networks (STGCNs). Our experimental results demonstrated that the combined STGCN and CNN model outperformed the standalone STGCN and CNN models. The findings of this study contribute to the advancement of short-term traffic speed prediction models, thereby improving road network flow management.

On the generalization discrepancy of spatiotemporal dynamics-informed graph convolutional networks

On the generalization discrepancy of spatiotemporal dynamics-informed graph convolutional networks

Towards multimodal graph neural networks for surgical instrument anticipation

PurposeDecision support systems and context-aware assistance in the operating room have emerged as the key clinical applications supporting surgeons in their daily work and are generally based on single modalities. The model- and knowledge-based integration of multimodal data as a basis for decision support systems that can dynamically adapt to the surgical workflow has not yet been established. Therefore, we propose a knowledge-enhanced method for fusing multimodal data for anticipation tasks.MethodsWe developed a holistic, multimodal graph-based approach combining imaging and non-imaging information in a knowledge graph representing the intraoperative scene of a surgery. Node and edge features of the knowledge graph are extracted from suitable data sources in the operating room using machine learning. A spatiotemporal graph neural network architecture subsequently allows for interpretation of relational and temporal patterns within the knowledge graph. We apply our approach to the downstream task of instrument anticipation while presenting a suitable modeling and evaluation strategy for this task.ResultsOur approach achieves an F1 score of 66.86% in terms of instrument anticipation, allowing for a seamless surgical workflow and adding a valuable impact for surgical decision support systems. A resting recall of 63.33% indicates the non-prematurity of the anticipations.ConclusionThis work shows how multimodal data can be combined with the topological properties of an operating room in a graph-based approach. Our multimodal graph architecture serves as a basis for context-sensitive decision support systems in laparoscopic surgery considering a comprehensive intraoperative operating scene.

Differences in spatiotemporal brain network dynamics of Montessori and traditionally schooled students

Across development, experience has a strong impact on the way we think and adapt. School experience affects academic and social-emotional outcomes, yet whether differences in pedagogical experience modulate underlying brain network development is still unknown. In this study, we compared the brain network dynamics of students with different pedagogical backgrounds. Specifically, we characterized the diversity and stability of brain activity at rest by combining both resting-state fMRI and diffusion-weighted structural imaging data of 87 4–18 years old students experiencing either the Montessori pedagogy (i.e., student-led, trial-and-error pedagogy) or the traditional pedagogy (i.e., teacher-led, test-based pedagogy). Our results revealed spatiotemporal brain dynamics differences between students as a function of schooling experience at the whole-brain level. Students from Montessori schools showed overall higher functional integration (higher system diversity) and neural stability (lower spatiotemporal diversity) compared to traditionally schooled students. Higher integration was explained mainly through the cerebellar (CBL) functional network. In contrast, higher temporal stability was observed in the ventral attention, dorsal attention, somatomotor, frontoparietal, and CBL functional networks. This study suggests a form of experience-dependent dynamic functional connectivity plasticity, in learning-related networks.

Human movement science-informed multi-task spatio temporal graph convolutional networks for fitness action recognition and evaluation

In recent years, with the rise of health consciousness, people’s demand for fitness has steadily increased. Utilizing automated human action recognition technology to monitor users’ movements during exercise continuously would help prevent situations where incorrect movements lead to injuries while working out. Skeleton-based human action recognition methods can overcome the susceptibility of past color-based and depth-based methods to various external backgrounds and noise, becoming a more successful solution in recent years. In this study, the auto-fitness advisor system we propose not only identifies the category of the action but also assesses the quality of the action and provides suggestions. We integrate human movement science, such as the Five Primary Kinetic Chains (5PKC), which defines the primary physiological principles in human movement, to enhance the accuracy of fitness action recognition by providing a more precise relationship between the human skeleton and muscles. For assessing the quality of movements and providing suggestions, we have designed a multi-task objective function within our model. Overall, the proposed model is a multi-task model based on Spatio Temporal Graph Convolutional Networks (ST-GCN), which employs the Five Primary Kinetic Chains (5PKC) as a partitioning strategy for skeletal information. In our experiments, we not only collected a certain amount of datasets in gyms to validate the performance of our model but also compared it with other current methods using existing public datasets.

A road network traffic flow data imputation method based on the fusion of spatiotemporal features and adversarial networks

In response to the problem of missing traffic flow data on highways, to solve the problem of insufficient mining of traffic flow characteristics using existing spatiotemporal correlation repair methods, a missing data repair method based on spatiotemporal fusion adversarial network is proposed based on analyzing the spatiotemporal characteristics of traffic flow, this method utilizes the fusion of GRU and GCN to capture the fine-grained spatiotemporal relationships of traffic flow, optimizes the generator and discriminator of the adversarial network, and achieves accurate repair of missing data. Experiments based on real data have shown that under various missing modes and rates, the model can learn the topological relationships of the road network, capture the temporal regularity and spatiotemporal correlation in the data, and effectively impute missing data.

Spatiotemporal Dynamic Multi-Hop Network for Traffic Flow Forecasting

Accurate traffic flow forecasting is vital for intelligent transportation systems, especially with urbanization worsening traffic congestion, which affects daily life, economic growth, and the environment. Precise forecasts aid in managing and optimizing transportation systems, reducing congestion, and improving air quality by cutting emissions. However, predicting outcomes is difficult due to intricate spatial relationships, nonlinear temporal patterns, and the challenges associated with long-term forecasting. Current research often uses static graph structures, overlooking dynamic and long-range dependencies. To tackle these issues, we introduce the spatiotemporal dynamic multi-hop network (ST-DMN), a Seq2Seq framework. This model incorporates spatiotemporal convolutional blocks (ST-Blocks) with residual connections in the encoder to condense historical traffic data into a fixed-dimensional vector. A dynamic graph represents time-varying inter-segment relationships, and multi-hop operation in the encoder’s spatial convolutional layer and the decoder’s diffusion multi-hop graph convolutional gated recurrent units (DMGCGRUs) capture long-range dependencies. Experiments on two real-world datasets METR-LA and PEMS-BAY show that ST-DMN surpasses existing models in three metrics.

Three Novel Artificial Neural Network Architectures Based on Convolutional Neural Networks for the Spatio-Temporal Processing of Solar Forecasting Data

In this work, three new convolutional neural network models—spatio-temporal convolutional neural network versions 1 and 2 (ST_CNN_v1 and ST_CNN_v2), and the spatio-temporal dilated convolutional neural network (ST_Dilated_CNN)—are proposed for solar forecasting and processing global horizontal irradiance (GHI) data enriched with meteorological and astronomical variables. A comparative analysis of the proposed models with two traditional benchmark models shows that the proposed ST_Dilated_CNN model outperforms the rest in capturing long-range dependencies, achieving a mean absolute error of 31.12 W/m2, a mean squared error of 54.07 W/m2, and a forecast skill of 37.21%. The statistical analysis carried out on the test set suggested highly significant differences in performance (p-values lower than 0.001 for all metrics in all the considered scenarios), with the model with the lowest variability in performance being ST_CNN_v2. The statistical tests applied confirmed the robustness and reliability of the proposed models under different conditions. In addition, this work highlights the significant influence of astronomical variables on prediction performance. The study also highlights the intricate relationship between the proposed models and meteorological and astronomical input characteristics, providing important insights into the field of solar prediction and reaffirming the need for further research into variability factors that affect the performance of models.

Dynamic Spatio-Temporal Adaptive Graph Convolutional Recurrent Networks for Vacant Parking Space Prediction

The prediction of vacant parking spaces (VPSs) can reduce the time drivers spend searching for parking, thus alleviating traffic congestion. However, previous studies have mostly focused on modeling the temporal features of VPSs using historical data, neglecting the complex and extensive spatial characteristics of different parking lots within the transportation network. This is mainly due to the lack of direct physical connections between parking lots, making it challenging to quantify the spatio-temporal features among them. To address this issue, we propose a dynamic spatio-temporal adaptive graph convolutional recursive network (DSTAGCRN) for VPS prediction. Specifically, DSTAGCRN divides VPS data into seasonal and periodic trend components and combines daily and weekly information with node embeddings using the dynamic parameter-learning module (DPLM) to generate dynamic graphs. Then, by integrating gated recurrent units (GRUs) with the parameter-learning graph convolutional recursive module (PLGCRM) of DPLM, we infer the spatio-temporal dependencies for each time step. Furthermore, we introduce a multihead attention mechanism to effectively capture and fuse the spatio-temporal dependencies and dynamic changes in the VPS data, thereby enhancing the prediction performance. Finally, we evaluate the proposed DSTAGCRN on three real parking datasets. Extensive experiments and analyses demonstrate that the DSTAGCRN model proposed in this study not only improves the prediction accuracy but can also better extract the dynamic spatio-temporal characteristics of available parking space data in multiple parking lots.

Investigating the influence of energy dynamics on price forecasts in tropical and traditional agricultural futures markets

ABSTRACT This research aims to investigate the intricate dynamics between energy futures prices and both tropical and traditional commodities futures prices, with a particular focus on the critical role of oil futures prices in predicting the trajectory of agricultural futures prices. The study employs the Dynamic Conditional Correlation-Multivariate Generalized Autoregressive Conditional Heteroskedasticity (DCC-MGARCH) model to analyse the interplay among oil and agricultural futures price series over 2008:01–2021:06. Additionally, it incorporates the innovative Spatio-temporal Information Recombination Hypergraph Neural Network (STIR-HGNN) model to highlight the differences in how energy futures prices predict tropical and traditional agricultural futures prices. The findings reveal numerous connections between oil prices and both tropical and traditional agricultural futures prices, underscoring the significant role of oil prices in forecasting agricultural futures price movements. The empirical insights from this study provide valuable guidance for futures market participants, encouraging them to use these findings to refine and optimize their market strategies, thus enhancing their ability to navigate and capitalize on the complexities of these interconnected markets.

Global–local spatio-temporal graph convolutional networks for video summarization

Video summarization aims to create concise and accurate summary to enable users to quickly grasp the key content of the original video for facilitating efficient video browsing. Most existing video summarization methods mainly employ recurrent neural networks to capture long-term dependencies in videos, yielding remarkable results. Nevertheless, these methods overlook the potential spatial features inside the video when modeling the video. To tackle this issue, we introduce a global–local spatio-temporal graph convolutional networks for video summarization (GL-STGCN). Initially inspired by the concept of 3-D convolution, we segment the video into non-overlapping segments to capture localized spatial features of sequential frames. Subsequently, a spatial graph is constructed for each segment, with a fixed time interval between neighboring spatial graphs. Then we use the pooling method to randomly delete the redundant nodes in the graph. We then leverage a temporal gating convolutional network to extract the global temporal relationships within the video. Employing the spatial features, a spatial graph convolutional network is utilized to capture the spatial connections among frames. As the graph node information evolves, the node features furnish a more precise depiction of the video content. Consequently, we employ the temporal gating convolutional network once more to refine the global temporal relations within the video. Extensive experiments on two public datasets are conducted in this paper, showing that our proposed method outperforms most state-of-the-art video summarization methods in terms of performance. Experimental results demonstrate the effectiveness of integrating global temporal and local spatial relationships.

Symmetric spatiotemporal learning network with sparse meter graph for short-term energy-consumption prediction in manufacturing systems

Short-term energy-consumption prediction is the basis of anomaly detection, real-time scheduling, and energy-saving control in manufacturing systems. Most existing methods focus on single-node energy-consumption prediction and suffer from difficult parameter collection and modelling. Although several methods have been presented for multinode energy-consumption prediction, their prediction performance needs to be improved owing to a lack of appropriate knowledge guidance and learning networks for complex spatiotemporal relationships. This study presents a symmetric spatiotemporal learning network (SSTLN) with a sparse meter graph (SMG) (SSTLN-SMG) that aims to predict multiple nodes based on energy-consumption time series and general process knowledge. The SMG expresses process knowledge by abstracting production nodes, material flows, and energy usage, and provides initial guidance for the SSTLN to extract spatial features. SSTLN, a symmetrical stack of graph convolutional networks (GCN) and gated linear units (GLU), is devised to achieve a trade-off not only between spatial and temporal feature extraction but also between detail capture and noise suppression. Extensive experiments were performed using datasets from an aluminium profile plant. The experimental results demonstrate that the proposed method allows multinode energy-consumption prediction with less prediction error than state-of-the-art methods, methods with deformed meter graphs, and methods with deformed learning networks.

Multi-site solar irradiance prediction based on hybrid spatiotemporal graph neural network

Constructing accurate spatiotemporal correlations is a challenging task in joint prediction of multiple photovoltaic sites. Some advanced algorithms for incorporating other surrounding site information have been proposed, such as graph neural network-based methods, which are usually based on static or dynamic graphs to build spatial dependencies between sites. However, the possibility of the simultaneous existence of multiple spatial dependencies is not considered. This paper establishes a spatiotemporal prediction model based on hybrid spatiotemporal graph neural network. In this model, we apply adaptive hybrid graph learning to learn composite spatial correlations among multiple sites. A temporal convolution module with multi-subsequence temporal data input is used to extract local semantic information to better predict future nonlinear temporal dependencies. A spatiotemporal adaptive fusion module is added to address the issue of integrating diverse spatiotemporal trends among multiple sites. To assess the model's predictive performance, nine solar radiation observation stations were selected in two different climatic environments. The average root mean square error (RMSE) of the constructed model was 38.51 and 49.90 W/m2, with average mean absolute error (MAE) of 14.72 and 23.06 W/m2, respectively. Single-site and multi-site prediction models were selected as baseline models. Compared with the baseline models, the RMSE and MAE reduce by 3.1%–20.8% and 8.9%–32.8%, respectively, across all sites. The proposed model demonstrates the effectiveness of improving accuracy in forecasting solar irradiance through multi-site predictions.

Prediction of three-dimensional ocean temperature in the South China Sea based on time series gridded data and a dynamic spatiotemporal graph neural network

Prediction of three-dimensional ocean temperature in the South China Sea based on time series gridded data and a dynamic spatiotemporal graph neural network

Identification and prediction method for acoustic emission and electromagnetic radiation signals of rock burst based on deep learning

With the deep development of underground rock engineering, the threat of rock burst disasters is increasing. At present, the identification and prediction of rock burst mostly rely on the experience of field staff to determine the critical value and development trend, and there is a lack of efficient and intelligent methods for the utilization of massive data. Therefore, this paper constructs a rock burst signal recognition and prediction model based on deep learning methods to solve the above problems. In this paper, the acoustic emission (AE) and electromagnetic radiation (EMR) data of the site are first marked and input into the long-short-term memory-fully connected neural network model to realize the identification of rock burst danger signals. Then, the graph data of the AE and EMR sensor monitoring networks are constructed and input into the spatiotemporal graph convolutional network signal prediction model to predict future monitoring data. Finally, this paper uses the same dataset to compare and analyze several other commonly used deep learning models. The results show that the model constructed in this paper has the best performance in the identification and prediction of AE and EMR signals with rockburst risk. This study can provide theoretical reference for intelligent monitoring and early warning of rock burst in underground rock engineering.

Bayesian Spatio-Temporal Graph Convolutional Network for Railway Train Delay Prediction

Bayesian Spatio-Temporal Graph Convolutional Network for Railway Train Delay Prediction

Glimpse and Zoom: Spatio-Temporal Focused Dynamic Network for Skeleton-Based Action Recognition

Glimpse and Zoom: Spatio-Temporal Focused Dynamic Network for Skeleton-Based Action Recognition