Spatio-temporal Network Research Articles

Intelligent Analysis of E-commerce Reviews based on Complex-Valued Spatio-temporal Graph Convolutional Neural Network Optimized Tyrannosaurus Optimization Algorithm

Intelligent analysis of E-commerce reviews based on multimodal social media big data using applies the benefit of innovative artificial intelligence method, especially deep learning algorithms, to study and realize web-based product consider and social network data connected to e-commerce. In this manuscript, Intelligent Analysis of E-commerce reviews based on Complex-Valued Spatio-temporal Graph Convolutional Neural Network Optimized Tyrannosaurus Optimization Algorithm (IAER-CVSGCNN-TOA)is proposed for Intelligent Analysis of E-commerce Reviews. The input data are collected from E-commerce big Dataset. Then data are pre-processing utilizing federated neural collaborative filtering (FNCF) to remove the noise and clean the data. The pre-processed data is provided to the CVSGCNN is used for intelligent analysis of e-commerce reviews. In general, CVSGCNN does no express adapting optimization approaches to determine optimal parameters to ensure accurate prediction. Hence, proposed to utilize the Tyrannosaurus Optimization Algorithm enhancement CVSGCNN for intelligent analysis of e-commerce reviews. The proposed IAER-CVSGCNN-TOA method is implemented on python. Then performance of proposed technique is analyzed with other existing techniques. The proposed technique attains 30.26%, 28.70% and 46.25% higher accuracy, 20.53%, 32.60%, and 53.43% higher precision, 25.23%, 23.15%, and 22.33%higher recall, 25.75%, 44.30% and 53.70%higher specificity comparing with the existing methods such as a Retracted: Online Troll Reviewer Detection Utilizing Deep Learning Methods (IAER-CNN), Developing an Intelligent System and DL Algorithms for Sentiment Analysis of E-Commerce Product Reviews(IAER-LSTM), Intelligent Perception System of Big Data Decision in Cross-Border e-Commerce Depend on Data Fusion(IAER-DFN) respectively.

Artistic Creation and Animation Scene Designing using Hierarchical Spatio-Temporal Graph Convolutional Neural Network Optimized with Gazelle Optimization Algorithm

Artistic Creation and Animation Scene Designing using Hierarchical Spatio-Temporal Graph Convolutional Neural Network Optimized with Gazelle Optimization Algorithm

Correlation-based feature selection and parallel spatiotemporal networks for efficient passenger flow forecasting in metro systems

This paper presents a novel framework for predicting metro passenger flow that is both interpretable and computationally efficient. The proposed method first uses a correlation-based spatiotemporal feature selection strategy (Cor-STFS) to identify the optimal input scheme for the prediction model, effectively reducing unnecessary interference. The framework then introduces a new multivariate passenger flow prediction architecture called STA-PTCN-BiGRU, which combines a spatiotemporal attention (STA) mechanism, parallel temporal convolutional networks (PTCN), and bidirectional gated recurrent units (BiGRU) to capture the dynamic internal patterns of passenger flow. By utilising parallel computing, this architecture significantly reduces resource consumption. The effectiveness of the proposed approach is evaluated using four datasets from the Shanghai Metro. Experimental results show that the new method outperforms baseline approaches in terms of root mean square error (RMSE), mean absolute error (MAE), and symmetric mean absolute percentage error (SMAPE), achieving average reductions of 9.98%, 8.08%, and 13.29% in these metrics, respectively.

OSTM-NET: Joint scale variation and occlusion handling deep network for real-time vehicle counting and volume estimation

Vehicle counting and traffic volume estimation using videos are difficult tasks crucial for efficient traffic control in smart cities. Several existing techniques rely in tracking and detecting mechanisms of the vehicles. These methods are ineffective for detecting occluded and small vehicles. Also, contextual information loss occurs in deep counting networks. An innovative Internet-of-Things (IoT)- driven Intelligent Transportation Management (ITM) system is proposed to address these issues. Initially, traffic videos are converted to temporal-spatial images instead of using complex detection and tracking methods. The density map for the temporal spatial image is estimated using an occlusion-aware spatio-temporal multi-scale network (OSTM-Net). It consists of two sub-networks for capturing occluded and small vehicles simultaneously. The scale-aware column network (SCNet) accurately captures small vehicles and preserves contextual information through enhanced scale representation. At the same time, the occlusion management network (OM-Net) uses position-sensitive regions of interest (PSRoI) deformable pooling to address the occlusion issues. Finally, volume estimation and counting are calculated in accordance with the density map obtained from OSTM-Net. Every path in the videos is processed separately using OSTM-Net to calculate the vehicle count in every path for effective traffic control in this proposed approach. Furthermore, the effectiveness of the sub-networks (SCNet and OM-Net) is validated using ablation experiments. The proposed IoT-based ITM achieves high performance in counting vehicles and estimating traffic volume compared to other existing approaches.

A Hybrid End-to-End Spatiotemporal Attention Neural Network With Graph-Smooth Signals for EEG Emotion Recognition

A Hybrid End-to-End Spatiotemporal Attention Neural Network With Graph-Smooth Signals for EEG Emotion Recognition

Spatiotemporal Network Based on GCN and BiGRU for Seizure Detection

Spatiotemporal Network Based on GCN and BiGRU for Seizure Detection

PDASTSGAT: An STSGAT-Based Multipath Data Scheduling Algorithm

How to select the transmitting path in MPTCP scheduling is an important but open problem. This paper proposes an intelligent data scheduling algorithm using spatiotemporal synchronous graph attention neural networks to improve MPTCP scheduling. By exploiting the spatiotemporal correlations in the data transmission process and incorporating graph self-attention mechanisms, the algorithm can quickly select the optimal transmission path and ensure fairness among similar links. Through simulations in NS3, the algorithm achieves a throughput gain of 7.9% compared to the PDAA3C algorithm and demonstrates improved packet transmission performance.

A spatiotemporal motion prediction network based on multi-level feature disentanglement

The prediction task is significantly challenged by the intricate scene information and motion variations present in spatiotemporal data. Existing prediction methods struggle to accurately forecast long-term outcomes, particularly for transient motions characterized by notable trends, such as hand lifts, jumps, or vehicle turns. To address these challenges, we introduce a Spatiotemporal Motion Prediction Network based on Multi-level Feature Disentanglement (FDPNet). The model delineates spatiotemporal prediction into two distinct stages: feature disentanglement and motion prediction. We first devise a Multi-level Feature Disentanglement (MFD) model to disentangle the multilayer features of motion within the temporal sequence, encompassing period, trend, and residual components. This disentanglement is based on disentangling spatiotemporal coupling, enabling the network to comprehensively grasp the genuine laws governing motion in the spatiotemporal evolution process. Second, to enhance the prediction accuracy of the network over extended periods, we introduce the Motion Differential Self-Attention LSTM unit (MDSA-LSTM). This unit employs differential operations to extract inter-frame motion trends, elevating the network's proficiency in capturing spatiotemporal correlations over long distances through an enhanced self-attention mechanism. FDPNet attains state-of-the-art performance on the Moving MNIST, UCF101, KITTI, and Caltech pedestrian datasets. These outcomes substantiate the substantial potential of this research within the realm of spatiotemporal prediction.

Multi-scale spatiotemporal attention network for neuron based motor imagery EEG classification

BackgroundIn recent times, the expeditious expansion of Brain-Computer Interface (BCI) technology in neuroscience, which relies on electroencephalogram (EEG) signals associated with motor imagery, has yielded outcomes that rival conventional approaches, notably due to the triumph of deep learning. Nevertheless, the task of developing and training a comprehensive network to extract the underlying characteristics of motor imagining EEG data continues to pose challenges. New methodThis paper presents a multi-scale spatiotemporal self-attention (SA) network model that relies on an attention mechanism. This model aims to classify motor imagination EEG signals into four classes (left hand, right hand, foot, tongue/rest) by considering the temporal and spatial properties of EEG. It is employed to autonomously allocate greater weights to channels linked to motor activity and lesser weights to channels not related to movement, thus choosing the most suitable channels. Neuron utilises parallel multi-scale Temporal Convolutional Network (TCN) layers to extract feature information in the temporal domain at various scales, effectively eliminating temporal domain noise. ResultsThe suggested model achieves accuracies of 79.26%, 85.90%, and 96.96% on the BCI competition datasets IV-2a, IV-2b, and HGD, respectively. Comparison with existing methodsIn terms of single-subject classification accuracy, this strategy demonstrates superior performance compared to existing methods. ConclusionThe results indicate that the proposed strategy exhibits favourable performance, resilience, and transfer learning capabilities.

Enhancing traffic flow prediction: Dual-branch graph convolutional network with rough data inference and adaptive spatial dependencies

Traffic flow prediction is a significant application of deep learning in spatio-temporal forecasting analysis. Existing research faces challenges that hinder prediction accuracy. One challenge is the inadequate capturing of spatial dependencies in traffic flow due to fixed pre-defined graph structures. Moreover, manually designed prior graphs still have limitations in extracting spatial features. Another challenge is the instability in short-term prediction performance when pre-defined graphs are completely abandoned in favor of parameter training. Additionally, ordinary RNN sequence convolution methods also struggle to capture long-term sequential patterns in large historical traffic data, leading to gradient vanishing or exploding issues. To address these challenges, we proposes a graph convolutional network for traffic flow prediction. We combine an improved prior graph with an adaptive graph to form a dual-branch spatio-temporal neural network. In the first branch, we introduce a time graph based on rough data inference to complement the predefined static graph. In the second branch, we construct an adaptive learning framework that dynamically learns the adjacency matrix and captures global road information. By utilizing enhanced multi-scale gated convolution, we extract spatio-temporal dependencies. Our method surpasses most baseline approaches according to extensive experiments conducted on public datasets.

Passenger spatiotemporal distribution prediction in airport terminals based on physics-guided spatio-temporal graph convolutional network and its effect on indoor environment prediction

The airport as an important transportation hub plays a leading role in promoting sustainable cities and new-type urbanization. To boost safe, environmental-friendly and technologically advanced airports, the passenger travel behavior as a core that decides the resource allocation, system tuning and capacity dispatching, must be grasped. Previous research in passenger distribution prediction focused on physics-based methods or only mining temporal dynamics. In this work, a refined passenger distribution prediction was modeled based on a learning-based method embedding physical prior knowledge, and then its effects on indoor environment prediction were analyzed. Among them, based on insect intelligent building architecture, a virtual spatial graph was defined in Guangzhou Baiyun International Airport Terminal 2, then a Wi-Fi positioning system was constructed; Next, a physics-guided spatio-temporal graph convolutional network, considering both the spatial dependencies and the passenger arrival pattern extracted from cost-free flight schedules, was developed for domestic and international passenger distribution predictions with R2 over 0.87 and 0.76 respectively; Lastly, the contributions of predicted occupant densities to the indoor environment prediction were evaluated with results showing that the average R2 for indoor temperature, relative humidity and CO2 concentration prediction was enhanced by 0.4 %∼91.5 %, 0.2 %∼29.7 % and 0.4 %∼45.4 % respectively as the prediction horizon broadening.

A spatio-temporal graph neural network for fall prediction with inertial sensors

Falls are the leading cause of unintentional human injury, having become a public health event of strong social concern. The fall prediction technology based on wearable inertial sensors is a relatively reliable solution in human activity monitoring, a user scenario with mobility and high information privacy sensitivity, and has the advantages of low cost, small size, and high precision. However, a key challenge in sensor-based fall prediction is the fact that a fall event can often occur in various configurations of fall poses together with their own spatio-temporal dependencies. This leads us to define a spatio-temporal model to explicitly characterize these internal configurations of poses. In particular, we introduce a graph neural network with spatio-temporal topological structure to encode such latent relations among poses by capturing representative patterns in fall events. Moreover, a human body orientation estimator is devised to represent human low limbs information and as a result, separate pose dependencies are globally consistent. Empirical evaluations on two benchmark datasets and one in-house dataset suggest our approach significantly outperforms the state-of-the-art methods.

MPSTAN: Metapopulation-Based Spatio-Temporal Attention Network for Epidemic Forecasting.

Accurate epidemic forecasting plays a vital role for governments to develop effective prevention measures for suppressing epidemics. Most of the present spatio-temporal models cannot provide a general framework for stable and accurate forecasting of epidemics with diverse evolutionary trends. Incorporating epidemiological domain knowledge ranging from single-patch to multi-patch into neural networks is expected to improve forecasting accuracy. However, relying solely on single-patch knowledge neglects inter-patch interactions, while constructing multi-patch knowledge is challenging without population mobility data. To address the aforementioned problems, we propose a novel hybrid model called metapopulation-based spatio-temporal attention network (MPSTAN). This model aims to improve the accuracy of epidemic forecasting by incorporating multi-patch epidemiological knowledge into a spatio-temporal model and adaptively defining inter-patch interactions. Moreover, we incorporate inter-patch epidemiological knowledge into both model construction and the loss function to help the model learn epidemic transmission dynamics. Extensive experiments conducted on two representative datasets with different epidemiological evolution trends demonstrate that our proposed model outperforms the baselines and provides more accurate and stable short- and long-term forecasting. We confirm the effectiveness of domain knowledge in the learning model and investigate the impact of different ways of integrating domain knowledge on forecasting. We observe that using domain knowledge in both model construction and the loss function leads to more efficient forecasting, and selecting appropriate domain knowledge can improve accuracy further.

SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-Supervised Skeleton-Based Action Recognition

Contrastive learning has achieved great success in skeleton-based action recognition. However, most existing approaches encode the skeleton sequences as entangled spatiotemporal representations and confine the contrasts to the same level of representation. Instead, this paper introduces a novel contrastive learning framework, namely Spatiotemporal Clues Disentanglement Network (SCD-Net). Specifically, we integrate the decoupling module with a feature extractor to derive explicit clues from spatial and temporal domains respectively. As for the training of SCD-Net, with a constructed global anchor, we encourage the interaction between the anchor and extracted clues. Further, we propose a new masking strategy with structural constraints to strengthen the contextual associations, leveraging the latest development from masked image modelling into the proposed SCD-Net. We conduct extensive evaluations on the NTU-RGB+D (60&120) and PKU-MMD (I&II) datasets, covering various downstream tasks such as action recognition, action retrieval, transfer learning, and semi-supervised learning. The experimental results demonstrate the effectiveness of our method, which outperforms the existing state-of-the-art (SOTA) approaches significantly. Our code and supplementary material can be found at https://github.com/cong-wu/SCD-Net.

Spatio-Temporal Pivotal Graph Neural Networks for Traffic Flow Forecasting

Traffic flow forecasting is a classical spatio-temporal data mining problem with many real-world applications. Recently, various methods based on Graph Neural Networks (GNN) have been proposed for the problem and achieved impressive prediction performance. However, we argue that the majority of existing methods disregarding the importance of certain nodes (referred to as pivotal nodes) that naturally exhibit extensive connections with multiple other nodes. Predicting on pivotal nodes poses a challenge due to their complex spatio-temporal dependencies compared to other nodes. In this paper, we propose a novel GNN-based method called Spatio-Temporal Pivotal Graph Neural Networks (STPGNN) to address the above limitation. We introduce a pivotal node identification module for identifying pivotal nodes. We propose a novel pivotal graph convolution module, enabling precise capture of spatio-temporal dependencies centered around pivotal nodes. Moreover, we propose a parallel framework capable of extracting spatio-temporal traffic features on both pivotal and non-pivotal nodes. Experiments on seven real-world traffic datasets verify our proposed method's effectiveness and efficiency compared to state-of-the-art baselines.

CI-STHPAN: Pre-trained Attention Network for Stock Selection with Channel-Independent Spatio-Temporal Hypergraph

Quantitative stock selection is one of the most challenging FinTech tasks due to the non-stationary dynamics and complex market dependencies. Existing studies rely on channel mixing methods, exacerbating the issue of distribution shift in financial time series. Additionally, complex model structures they build make it difficult to handle very long sequences. Furthermore, most of them are based on predefined stock relationships thus making it difficult to capture the dynamic and highly volatile stock markets. To address the above issues, in this paper, we propose Channel-Independent based Spatio-Temporal Hypergraph Pre-trained Attention Networks (CI-STHPAN), a two-stage framework for stock selection, involving Transformer and HGAT based stock time series self-supervised pre-training and stock-ranking based downstream task fine-tuning. We calculate the similarity of stock time series of different channel in dynamic intervals based on Dynamic Time Warping (DTW), and further construct channel-independent stock dynamic hypergraph based on the similarity. Experiments with NASDAQ and NYSE markets data over five years show that our framework outperforms SOTA approaches in terms of investment return ratio (IRR) and Sharpe ratio (SR). Additionally, we find that even without introducing graph information, self-supervised learning based on the vanilla Transformer Encoder also surpasses SOTA results. Notable improvements are gained on the NYSE market. It is mainly attributed to the improvement of fine-tuning approach on Information Coefficient (IC) and Information Ratio based IC (ICIR), indicating that the fine-tuning method enhances the accuracy and stability of the model prediction.

Advancing storm surge forecasting from scarce observation data: A causal-inference based Spatio-Temporal Graph Neural Network approach

Rapid and precise forecasting of storm surge in coastal regions is crucial for ensuring safety of coastal communities’ life and property. Yet, learning a data-driven forecasting model from observation data such as gauges and post-event reconnaissance remains challenging, due to the observation data scarcity and the real-world complexity. Recently, deep learning has received increasing attention, but existing deep learning approaches solely focus on individual site scenarios, ignoring the value of information contained in neighboring sites’ observations. In this study, we propose to integrate graph neural networks (GNN) and gated recurrent unit (GRU) to capture the spatial and temporal storm surge dependencies across multiple observation stations. GNN provides the unique capability to model non-Euclidean complex spatial relationship across observation stations, while GRU enhances the data efficiency of temporal dependency modeling. To account for the effect of complex coastline topography, the Liang–Kleeman information flow theory is employed to establish a causal-inference based graph edge scheme connecting multiple observation stations. The Causal-inference based Spatio-Temporal Graph Neural Network (CSTGNN) were trained and evaluated on 13-year observation data from 4 observation stations along Florida coastline. Experiments affirm the competence of CSTGNN, which outperformed six commonly used competitive baselines across different metrics and observation stations, under lead times up to six hours. Furthermore, benefits of capturing the spatial dependency and leveraging causal inference are also comprehensively examined. To conclude, we believe that this novel spatio-temporal forecasting framework can result in enhanced coastal resilience by its improved storm surge forecasting capability.

Deep spatiotemporal fusion network for vision-based robotic inspection of structures

The convolutional neural networks commonly deployed for semantic understanding of visual inspection data can, in general, learn robust spatial features. However, they lack the ability to capture temporal dependencies that characterize the video data collected by various robotic inspection systems. As a result, they are found lacking in dealing with various challenges arising from cross-view illumination variation, perspective difference, scale change, background clutter, and occlusion. Their performance is further deteriorated by motion blur and other distortions induced by rapid camera movement. This study aims to address this challenge by extending the task of visual scene understanding from the still image domain to the video domain by incorporating cross-frame information fusion. A deep end-to-end network is developed by integrating an encoder–decoder-based convolutional neural network with a long short-term memory-based recurrent neural network for pixel-level semantic labeling of sequential visual inspection data. The proposed multishot architecture can jointly learn discriminative fusion features leading to a rich understanding of the complex spatiotemporal dynamics. The proposed approach is validated with two case studies involving automatic structural element segmentation in robotic building and bridge inspection videos. Two different multishot fusion techniques are suggested leveraging sequence-to-one and sequence-to-sequence architectures. Additionally, two different fusion schemes based on the sum-of-scores and Bayesian updating rules are examined to aggregate multiple label maps produced at each time step by an overlapping sliding window-based inference scheme. A comprehensive performance evaluation indicated that multishot fusion could enhance the intersection over union (IoU) score by 4.6% and 13.3% for building and bridge component segmentation tasks, respectively, compared to a baseline single-shot approach.

High Spatiotemporal Resolution River Networks Mapping on Catchment Scale Using Satellite Remote Sensing Imagery and DEM Data

AbstractCharacterizing and understanding the changes in the flow regimes of rivers have been challenging. Existing global river network data sets are not updated and can only identify rivers wider than 30 m. We propose a novel automated method to map river networks on a monthly basin scale for the first time at 10‐m resolution using Sentinel‐1 Synthetic Aperture Radar, Sentinel‐2 multispectral images, and the AW3D30 Digital Surface Model. This method achieved an overall accuracy of 95.8%. The total length of the Yellow River network produced is 40,280 km, approximately 3.2 times that of the Global River Widths from Landsat (GRWL) database, more effectively covering small and medium rivers. The monthly river geometry revealed a positive correlation between river network area and precipitation. This study is expected to provide a cost‐effective alternative to accurately mapping global river networks and advance our understanding of the changes and drivers of river systems.

A novel spatio-temporal memory network for video anomaly detection

A novel spatio-temporal memory network for video anomaly detection