Spatial-temporal Dependencies Research Articles

PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction

As a core technology of Intelligent Transportation System, traffic flow prediction has a wide range of applications. The fundamental challenge in traffic flow prediction is to effectively model the complex spatial-temporal dependencies in traffic data. Spatial-temporal Graph Neural Network (GNN) models have emerged as one of the most promising methods to solve this problem. However, GNN-based models have three major limitations for traffic prediction: i) Most methods model spatial dependencies in a static manner, which limits the ability to learn dynamic urban traffic patterns; ii) Most methods only consider short-range spatial information and are unable to capture long-range spatial dependencies; iii) These methods ignore the fact that the propagation of traffic conditions between locations has a time delay in traffic systems. To this end, we propose a novel Propagation Delay-aware dynamic long-range transFormer, namely PDFormer, for accurate traffic flow prediction. Specifically, we design a spatial self-attention module to capture the dynamic spatial dependencies. Then, two graph masking matrices are introduced to highlight spatial dependencies from short- and long-range views. Moreover, a traffic delay-aware feature transformation module is proposed to empower PDFormer with the capability of explicitly modeling the time delay of spatial information propagation. Extensive experimental results on six real-world public traffic datasets show that our method can not only achieve state-of-the-art performance but also exhibit competitive computational efficiency. Moreover, we visualize the learned spatial-temporal attention map to make our model highly interpretable.

Trafformer: Unify Time and Space in Traffic Prediction

Traffic prediction is an important component of the intelligent transportation system. Existing deep learning methods encode temporal information and spatial information separately or iteratively. However, the spatial and temporal information is highly correlated in a traffic network, so existing methods may not learn the complex spatial-temporal dependencies hidden in the traffic network due to the decomposed model design. To overcome this limitation, we propose a new model named Trafformer, which unifies spatial and temporal information in one transformer-style model. Trafformer enables every node at every timestamp interact with every other node in every other timestamp in just one step in the spatial-temporal correlation matrix. This design enables Trafformer to catch complex spatial-temporal dependencies. Following the same design principle, we use the generative style decoder to predict multiple timestamps in only one forward operation instead of the iterative style decoder in Transformer. Furthermore, to reduce the complexity brought about by the huge spatial-temporal self-attention matrix, we also propose two variants of Trafformer to further improve the training and inference speed without losing much effectivity. Extensive experiments on two traffic datasets demonstrate that Trafformer outperforms existing methods and provides a promising future direction for the spatial-temporal traffic prediction problem.

Human Motion Prediction via Dual-Attention and Multi-Granularity Temporal Convolutional Networks

Intelligent devices, which significantly improve the quality of life and work efficiency, are now widely integrated into people’s daily lives and work. A precise understanding and analysis of human motion is essential for achieving harmonious coexistence and efficient interaction between intelligent devices and humans. However, existing human motion prediction methods often fail to fully exploit the dynamic spatial correlations and temporal dependencies inherent in motion sequence data, which leads to unsatisfactory prediction results. To address this issue, we proposed a novel human motion prediction method that utilizes dual-attention and multi-granularity temporal convolutional networks (DA-MgTCNs). Firstly, we designed a unique dual-attention (DA) model that combines joint attention and channel attention to extract spatial features from both joint and 3D coordinate dimensions. Next, we designed a multi-granularity temporal convolutional networks (MgTCNs) model with varying receptive fields to flexibly capture complex temporal dependencies. Finally, the experimental results from two benchmark datasets, Human3.6M and CMU-Mocap, demonstrated that our proposed method significantly outperformed other methods in both short-term and long-term prediction, thereby verifying the effectiveness of our algorithm.

A Multi-Level Attentive Context-Aware Trajectory Prediction Algorithm for Mobile Social Users

The prediction of a user’s trajectory is a key problem in mobility prediction, which has been applied to a range of fields such as location-based service recommendations and traffic planning. The impact of users’ social contacts on mobility is not adequately considered in the current trajectory prediction research. Furthermore, the spatial–temporal dependence of long trajectories is difficult to characterize by conventional recurrent neural network models. A multi-level attentive context-aware trajectory prediction model (MACTP) for mobile social users is proposed in this research to address the above problems. Specifically, users’ social preferences are captured by friend-level attention, and different friends are allocated varying weights. The impact of other check-in points in the trajectory on the present check-in point is considered through check-in-level attention. Trajectory-level attention is used to obtain the representation of historical trajectories influenced by current trajectories, as well as the spatial–temporal dependencies of longer trajectories. Experimental results on two real-world datasets demonstrate that the proposed model significantly improves trajectory prediction performance.

Signal-control refined dynamic traffic graph model for movement-based arterial network traffic volume prediction

Forecasting the forthcoming intersection movement-based traffic volume enables adaptive traffic control systems to dynamically respond to the fluctuation of traffic demands. In this paper, a deep-learning based Signal-control Refined Dynamic Traffic Graph (ScR-DTG) Model is proposed for advancing the network-level movement-based traffic volume prediction task. The proposed model attempts to further improve the-state-of-art and practice algorithms in traffic prediction for arterial network adaptive signal control utilizing tradition traffic flow theory boosted deep-learning methodology. For precisely inferencing the movement-based demand at cycle-to-cycle level, the proposed model incorporates spatial graph convolution inferencing layer and temporal inferencing layer to explore both the intricate spatial temporal dependencies, respectively. A signal control refining module is contrived to deduce the controlled movement saturation flow and introduce the essential control inferences, which is of great significance but frequently neglected in the previous researches. Additionally, according to the real-time movement specified travel time, this paper creatively constructs an adjacent graph with dynamic order for more accurately capturing the ever-changing spatial relevancies. Field experiments with multiple signal schemes were conducted in the downtown area of Zhangzhou (China). The promising results demonstrated the state-of-the-art accuracy than other high-performance volume prediction algorithms. Implementing the proposed model enables to obtain accurate movement-based volume predictions, which would assist the traffic management agencies in adjusting signal timing adaptively and further improve the efficiency of signal intersection.

Channel attention-based spatial-temporal graph neural networks for traffic prediction

PurposeTraffic flow prediction has always been a top priority of intelligent transportation systems. There are many mature methods for short-term traffic flow prediction. However, the existing methods are often insufficient in capturing long-term spatial-temporal dependencies. To predict long-term dependencies more accurately, in this paper, a new and more effective traffic flow prediction model is proposed.Design/methodology/approachThis paper proposes a new and more effective traffic flow prediction model, named channel attention-based spatial-temporal graph neural networks. A graph convolutional network is used to extract local spatial-temporal correlations, a channel attention mechanism is used to enhance the influence of nearby spatial-temporal dependencies on decision-making and a transformer mechanism is used to capture long-term dependencies.FindingsThe proposed model is applied to two common highway datasets: METR-LA collected in Los Angeles and PEMS-BAY collected in the California Bay Area. This model outperforms the other five in terms of performance on three performance metrics a popular model.Originality/value(1) Based on the spatial-temporal synchronization graph convolution module, a spatial-temporal channel attention module is designed to increase the influence of proximity dependence on decision-making by enhancing or suppressing different channels. (2) To better capture long-term dependencies, the transformer module is introduced.

MVSTGN: A Multi-View Spatial-Temporal Graph Network for Cellular Traffic Prediction

Timely and accurate cellular traffic prediction is difficult to achieve due to the complex spatial-temporal characteristics of cellular traffic. The latest approaches mainly aim to model local spatial-temporal dependencies of cellular traffic based on deep learning techniques but lack the consideration of diverse global spatial-temporal correlations hidden in cellular traffic. To tackle this issue, we propose a novel multi-view spatial-temporal graph network (MVSTGN), which combines attention and convolution mechanisms into traffic pattern analysis, enabling the comprehensive excavation of spatial-temporal characteristics. Specifically, the MVSTGN realizes the above statement from three spatial-temporal views: 1) From a global spatial view, two spatial attention modules are proposed to capture the global spatial correlations between different regions at node and trend levels; 2) From a global temporal view, a temporal attention module is employed to capture and encode global temporal correlations between traffic at different times; 3) From a local spatial-temporal view, a dense convolution module is developed to further excavate the local spatial-temporal dependencies in cellular traffic. Consequently, a successful cellular traffic prediction strategy is constructed to fully explore the spatial-temporal characteristics from multiple views. The experimental results on a popular real-world cellular traffic dataset demonstrate that the MVSTGN achieves obvious improvements over baselines.

Self-Attention Graph Convolution Residual Network for Traffic Data Completion

Complete and accurate traffic data is critical in urban traffic management, planning and operation. In fact, real-world traffic data contains missing values due to multiple factors, such as device outages and communication errors. For traffic data completion task, most of the existing methods are matrix/tensor completion methods, which usually enforce low rank constraint on traffic data matrix/tensor. But they neglect the graph structure of traffic data, resulting in low completion performance. Recently, graph convolutional networks have achieved remarkable results in traffic data forecasting due to their abilities of feature extraction and nonlinear fitting on arbitrarily graph-structured data. However, there are few studies based on graph neural networks for traffic data completion task. In this paper, we propose a traffic data completion model based on graph convolutional network model to impute missing values from the perspective of deep learning. This model utilizes graph convolution to model the local spatial dependency. As for global spatial dependency and temporal dependency, this model incorporates self-attention mechanism, which is applied in the spatial and temporal dimensions respectively. The experimental results on the two real-time datasets demonstrate that the proposed model outperforms the baseline methods significantly under arbitrarily missing scenarios.

Attention correlated appearance and motion feature followed temporal learning for activity recognition

<span lang="EN-US">Recent advances in deep neural networks have been successfully demonstrated with fairly good accuracy for multi-class activity identification. However, existing methods have limitations in achieving complex spatial-temporal dependencies. In this work, we design two stream fusion attention (2SFA) connected to a temporal bidirectional gated recurrent unit (GRU) one-layer model and classified by prediction voting classifier (PVC) to recognize the action in a video. Particularly in the proposed deep neural network (DNN), we present 2SFA for capturing appearance information from red green blue (RGB) and motion from optical flow, where both streams are correlated by proposed fusion attention (FA) as the input of a temporal network. On the other hand, the temporal network with a bi-directional temporal layer using a GRU single layer is preferred for temporal understanding because it yields practical merits against six topologies of temporal networks in the UCF101 dataset. Meanwhile, the new proposed classifier scheme called PVC employs multiple nearest class mean (NCM) and the SoftMax function to yield multiple features outputted from temporal networks, and then votes their properties for high-performance classifications. The experiments achieve the best average accuracy of 70.8% in HMDB51 and 91.9%, the second best in UCF101 in terms of 2DConvNet for action recognition.</span>

On prediction of air pollutants with Takagi-Sugeno models based on a hierarchical clustering identification method

In recent years, air pollution has attracted considerable attention worldwide. As an effective air protection method, the accurate prediction of air pollutants can help provide an early warning against harmful air pollutants and plan air pollution prevention and control strategies. Due to its high prediction capability, a novel method using a Takagi-Sugeno fuzzy model is presented to predict air pollutants in this study. This work investigates fuzzy model identification based on a larger dataset, and a hierarchical clustering-based identification method with model structure selection is proposed. The novelty of this work is as follows: First, a novel hierarchical clustering method has been employed in air quality forecasting. This method improves the widely used BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) by introducing a refinement phase for handling clusters with arbitrary shapes. Second, a cluster validity measure automatically determines the number of clusters. Then, the estimation of the model order selection and the model parameters is formulated as a sparse optimization problem, which can be solved using global optimization. Finally, the proposed Takagi-Sugeno model can learn the local trend pattern and spatial-temporal dependencies of multivariate air quality-related time series data. The presented method is demonstrated with air quality measurements in Shanghai, and the results show that the proposed method can achieve acceptable prediction performance.

Pre-encoding based temporal dependent rate–distortion optimization for HEVC

Rate–distortion optimization (RDO) is a crucial technique in block-based hybrid video encoders, which determines the coding option for a certain coding unit to achieve optimal rate–distortion (R–D) performance. However, the current RDO method, which is applied in High Efficiency Video Coding (HEVC), is far from being globally optimal as it ignores the interaction among coding options. In fact, the current coding option not only determines the R–D point of the coded unit but also influences the achievable R–D​ performance of the subsequent coding units due to the spatial–temporal dependencies among the block/frame optimizations. In this paper, we first investigate the R–D characteristics, including operational R–D curves and R–D dependencies, in HEVC through several simulation experiments. It is shown that under the hierarchical fixed QP setting in the HEVC encoder, the temporal R–D dependence dominates the coding process, and the distortion of the reference frame has a strong impact on that of coded frames while less influence on rate. We then introduce distortion effect factors to quantitatively measure the temporal R–D​ dependency. The distortion effect factor at frame level reflects the correlation degree between the current coding frame and the previous coded frames, and the one at block level reflects the influence degree that the distortion of a block to that of subsequent blocks which refer to it in temporal domain. Finally, a pre-encoding based temporal dependent RDO is proposed, which adaptively adjusts quantization parameters and Lagrange multipliers according to the distortion effect factors. Experimental results show that under configurations of low-delay B and P frames, the proposed method can achieve 5.4% and 5.5% bit rate savings on average, respectively, with around 7% encoding complexity increment, compared with the original HEVC encoder.

Multi-view dynamic graph convolution neural network for traffic flow prediction

The rapid urbanization and continuous improvement of road traffic equipment result in massive daily production of traffic data. These data contain the long-term evolution of traffic flow and dynamic changes in the traffic road network. Due to the complex topology of the traffic road network, traffic flow prediction is challenging as it contains complex, multi-periodic patterns, and is often affected by sudden events. In this paper, we propose a Multi-View Dynamic Graph Convolution Network (MVDGCN) that captures different levels of spatial–temporal dependencies to predict traffic flow. Firstly, we use the coupling graph convolution network to learn the relationship matrix among stations dynamically, capturing the spatial dependencies at different levels in the traffic network. Secondly, we establish three encoder–decoders, representing hourly, daily, and weekly views, to extract the evolution law of traffic flow from three different time periods. Finally, we use the dynamic fusion module to merge the spatial–temporal dependencies extracted from the multi-view encoder–decoders. We conducted experiments on two real datasets, NYCTaxi and NYCBike, and found that our proposed MVDGCN model outperformed the best baseline, improving the RMSE, MAE, PCC, and MAPE by 12.9%, 6.2%, 0.8%, and 6.5% respectively on the NYCBike dataset and 9.2%, 4.2%, 4.6%, and 3.0% respectively on the NYCTaxi dataset. These results show that the proposed MVDGCN model performs better than state-of-the-art algorithms.

Transformer Memory for Interactive Visual Navigation in Cluttered Environments

Substantial progress has been achieved in embodied visual navigation based on reinforcement learning (RL). These studies presume that the environment is stationary where all the obstacles are static. However, in real cluttered scenes, interactable objects (e.g. shoes and boxes) blocking the way of robots makes the environment non-stationary. Accordingly, the ego-centric visual agent will easily get stuck in the dilemma of finding the next waypoint as it struggles to decide whether to push the obstacles ahead. To handle the predicament, we formulate this interactive visual navigation as Partial Observed Markov Decision Process (POMDP). As the transformer encoder has demonstrated its superior ability to capture the spatial-temporal dependencies in natural language processing. We propose a transformer-based memory to empower the agents utilizing the historical interactive information. However, directly leveraging the transformer architecture in the RL settings is highly unstable. We further propose a surrogate objective to predict the next waypoint as the auxiliary task, which facilitates the representation learning and bootstraps the RL. We demonstrate our method in the iGibson environment and experimental results show a significant improvement over the interactive Gibson benchmark and the related recurrent RL policy both in the validation seen scenes and the test unseen scenes.

Dynamic Spatial Correlation in Graph WaveNet for Road Traffic Prediction

Road traffic prediction is a crucial area currently investigated under the umbrella of the intelligent transportation system. Timely and accurate traffic prediction is a challenging problem because of the diverse nature of roads, abrupt changes in speed, and the existence of dependencies between road segments. A critical component of this research is addressing the dynamic spatial and non-linear temporal dependencies in the road network. The traffic conditions in a traffic road network change continuously, and for precise predictions, time-varying spatial correlation needs to be integrated into the model. This study intends to incorporate dynamic spatial dependencies in the Graph WaveNet model by applying attention mechanisms that can compute attention scores for the self-adaptive adjacency matrix in the time domain. We compared the computation cost of our proposed model of a graph attention network with multi-head attention with Graph WaveNet for up to 60 min. Our model gave the best result for 60-min prediction with an average percentage decrease of 3.4% and 4.76% in root-mean-square error on PEMS-BAY and METR-LA datasets, respectively. However, the model training time is increased because of the added computation of attention scores.

A difference attention ResNet-LSTM network for epileptic seizure detection using EEG signal

Epileptic seizures can affect the patient’s physical function and cause irreversible damage to their brain. It is vital to detect epilepsy seizures in time and give patients antiepileptic medical treatment. Hybrid deep learning models, which combine convolutional neural network and recurrent neural network, have better epileptic seizure detection performance as they could simultaneously extract spatial and temporal features. However, the existing hybrid deep learning models still have the following two weaknesses. Firstly, they directly input the raw electroencephalogram signals, where the epilepsy seizure information is limited. Secondly, some characteristic information is extracted in the feature map, distracting the attention of deep learning model. To address these issues, this paper proposes a difference attention ResNet-LSTM network (DARLNet). The proposed model uses a residual neural network (ResNet) and a long short-term memory network (LSTM) to capture spatial correlations and temporal dependencies, respectively. Besides, a difference layer is developed to automatically mine additional epileptic seizure information. Moreover, the channel attention module is introduced to make the model focus on seizure-relevant information. Several groups of experiments are conducted to evaluate the performance of DARLNet based on the Bonn Electroencephalogram dataset, which verifies the superiority of DARLNet on the two-category and five-category epileptic seizure detection tasks.

DSTAGCN: Dynamic Spatial-Temporal Adjacent Graph Convolutional Network for Traffic Forecasting

Capturing complex and dynamic spatial-temporal dependencies of traffic data is of great importance for accurate and real-time traffic forecasting in intelligent transportation systems. The spatial-temporal dependency between traffic locations is often dynamic, which means the correlation between the traffic status of different locations changes jointly over their spatial distance and the time slice they are in. Most of the existing Graph Convolutional Network-based methods usually capture spatial and temporal dependencies separately and then combine them in a parallel or serial mechanism to capture the spatial-temporal features. They always utilize the predefined static graph structure to capture both local correlations and global dependencies in the same time slice. These methods are incapable of directly learning dynamic spatial-temporal dependency across time slices. Meanwhile, it is challenging to learn the spatiotemporal correlation knowledge among traffic locations only by using neural networks. To address these issues, we propose a novel Dynamic Spatial-Temporal Adjacent Graph Convolutional Network (DSTAGCN), which connects the latest time slice with each past time slice to construct the spatial-temporal graph. DSTAGCN can directly learn the global spatial dependency across time and simultaneously capture the spatial-temporal dependencies through graph convolution. Since fuzzy theory make it possible to represent uncertain relationships, a simplified fuzzy neural network that integrates fuzzy systems and neural networks is designed to help generating the graph adjacency matrix representing the dynamic adjacency correlations. Experiments on public datasets show that our method outperforms baselines with fast convergence.

GraphSTGAN: Situation understanding network of slow-fast high maneuvering targets for maritime monitor services of IoT data

With the rapid growth of the maritime Internet of Things (IoT) devices for Maritime Monitor Services (MMS), maritime traffic controllers could not handle a massive amount of data in time. For unmanned MMS, one of the key technologies is situation understanding. However, the presence of slow-fast high maneuvering targets and track breakages due to radar blind zones make modeling the dynamics of marine multi-agents difficult, and pose significant challenges to maritime situation understanding. In order to comprehend the situation accurately and thus offer unmanned MMS, it is crucial to model the complex dynamics of multi-agents using IoT big data. Nevertheless, previous methods typically rely on complex assumptions, are plagued by unstructured data, and disregard the interactions between multiple agents and the spatial-temporal correlations. A deep learning model, Graph Spatial-Temporal Generative Adversarial Network(GraphSTGAN), is proposed in this paper, which uses graph neural network to model unstructured data and uses STGAN to learn the spatial-temporal dependencies and interactions. Extensive experiments show the effectiveness and robustness of the proposed method.

Learning Spatial-Temporal Dynamics for Short-Term Passenger Flow Prediction in Urban Rail Transit

Accurate short-term passenger flow prediction in urban rail transit (URT) plays an important role in ensuring the stable operation of the URT systems. Because of the complex dynamic spatial-temporal dependencies and potential semantic correlations of the URT network, accurate and effective short-term passenger flow prediction is challenging. To solve these problems, a novel model called the dynamic spatial-temporal graph convolutional network (DSTGCN) was proposed. Firstly, spatial semantic graphs (SSGs) were established to encode the spatial dependencies and semantic correlations of the URT network. Meanwhile, the dynamic graph convolutional network (DGCN) with the spatial attention mechanism was used to learn the dynamic spatial correlations of the nodes in the SSGs. Then, the long short-term memory (LSTM) network was integrated into the DGCN to learn the dynamic changes of passenger flow and capture local temporal dependencies. Moreover, the temporal attention mechanism was introduced after LSTM to capture global dynamic temporal correlations by adjusting the weights of different sequence information. Finally, the full connection layers were used to output the prediction results. Several experiments were conducted on Nanning Metro Line 1 real datasets to evaluate the model. The experimental results showed that the DSTGCN can effectively capture the dynamic spatial-temporal dependencies and semantic associations of the passenger flow. Besides, the prediction performances of the DSTGCN were better than those of existing baseline models, and it can provide technical support for improving the intelligent planning and operation decisions of URT systems.

Long Short-Term Fusion Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting

Traffic flow forecasting, as one of the important components of intelligent transport systems (ITS), plays an indispensable role in a wide range of applications such as traffic management and city planning. However, complex spatial dependencies and dynamic changes in temporal patterns exist between different routes, and obtaining as many spatial-temporal features and dependencies as possible from node data has been a challenging task in traffic flow prediction. Current approaches typically use independent modules to treat temporal and spatial correlations separately without synchronously capturing such spatial-temporal correlations, or focus only on local spatial-temporal dependencies, thereby ignoring the implied long-term spatial-temporal periodicity. With this in mind, this paper proposes a long-term spatial-temporal graph convolutional fusion network (LSTFGCN) for traffic flow prediction modeling. First, we designed a synchronous spatial-temporal feature capture module, which can fruitfully extract the complex local spatial-temporal dependence of nodes. Second, we designed an ordinary differential equation graph convolution (ODEGCN) to capture more long-term spatial-temporal dependence using the spatial-temporal graph convolution of ordinary differential equation. At the same time, by integrating in parallel the ODEGCN, the spatial-temporal graph convolution attention module (GCAM), and the gated convolution module, we can effectively make the model learn more long short-term spatial-temporal dependencies in the processing of spatial-temporal sequences.Our experimental results on multiple public traffic datasets show that our method consistently obtained the optimal performance compared to the other baselines.

Optimal Routing with Spatial-Temporal Dependencies for Traffic Flow Control in Intelligent Transportation Systems

In Intelligent Transportation Systems (ITS), controlling the traffic flow of a region in a city is the major challenge. Particularly, allocation of the traffic-free route to the taxi drivers during peak hours is one of the challenges to control the traffic flow. So, in this paper, the route between the taxi driver and pickup location or hotspot with the spatial-temporal dependencies is optimized. Initially, the hotspots in a region are clustered using the density-based spatial clustering of applications with noise (DBSCAN) algorithm to find the hot spots at the peak hours in an urban area. Then, the optimal route is allocated to the taxi driver to pick up the customer in the hotspot. Before allocating the optimal route, each route between the taxi driver and the hot spot is mapped to the number of taxi drivers. Among the map function, the optimal map is selected using the rain optimization algorithm (ROA). If more than one map function is obtained as the optimal solution, the map between the route and the taxi driver who has done the least number of trips in the day is chosen as the final solution This optimal route selection leads to control of the traffic flow at peak hours. Evaluation of the approach depicts that the proposed traffic flow control scheme reduces traveling time, waiting time, fuel consumption, and emission.