Spatio-temporal Neural Network Model Research Articles

A macro–micro spatio-temporal neural network for traffic prediction

Accurate traffic prediction is crucial for planning, management and control of intelligent transportation systems. Most state-of-the-art methods for traffic prediction effectively capture complex traffic patterns (e.g. spatial and temporal correlations of traffic data) by employing spatio-temporal neural networks as prediction models, together with graph convolution networks to learn spatial correlations of prediction objects (e.g. traffic states of road segments, as in this study). Such spatial correlations can be regarded as micro correlations. However, there are also macro correlations between regions, each of which is composed of multiple road segments or artificially partitioned areas. Macro correlations represent another type of interaction within road segments, and should be carefully considered when predicting traffic. The diversity of micro spatial correlations and corresponding macro spatial correlations (e.g. correlations based on physical proximity or traffic pattern similarity) further increases the complexity of traffic prediction. We overcome these challenges by developing a macro–micro spatio-temporal neural network model, denoted ‘MMSTNet’. MMSTNet captures spatio-temporal patterns by (a) utilizing a graph convolution network and a spatial attention network to capture micro and macro spatial correlations, respectively; (b) employing a temporal convolution network and a temporal attention network to learn temporal patterns; and (c) integrating hierarchically learned representations based on designed attention mechanisms. We perform evaluations on two real-world datasets and thereby demonstrate that MMSTNet outperforms state-of-the-art models in traffic prediction tasks.

Assessing the impacts of past and ongoing deforestation on rainfall patterns in South America.

Despite recent advances in modeling forest-rainfall relationships, the current understanding of changes in observed rainfall patterns resulting from historical deforestation remains limited. To address this knowledge gap, we analyzed how 40 years of deforestation has altered rainfall patterns in South America as well as how current Amazonian forest cover sustains rainfall. First, we develop a spatiotemporal neural network model to simulate rainfall as a function of vegetation and climate inputs in South America; second, we assess the rainfall effects of observed deforestation in South America during the periods 1982-2020 and 2000-2020; third, we assess the potential rainfall changes in the Amazon biome under two deforestation scenarios. We find that, on average, cumulative deforestation in South America from 1982 to 2020 has reduced rainfall over the period 2016-2020 by 18% over deforested areas, and by 9% over non-deforested areas across South America. We also find that more recent deforestation, that is, from 2000 to 2020, has reduced rainfall over the period 2016-2020 by 10% over deforested areas and by 5% over non-deforested areas. Deforestation between 1982 and 2020 has led to a doubling in the area experiencing a minimum dry season of 4 months in the Amazon biome. Similarly, in the Cerrado region, there has been a corresponding doubling in the area with a minimum dry season of 7 months. These changes are compared to a hypothetical scenario where no deforestation occurred. Complete conversion of all Amazon forest land outside protected areas would reduce average annual rainfall in the Amazon by 36% and complete deforestation of all forest cover including protected areas would reduce average annual rainfall in the Amazon by 68%. Our findings emphasize the urgent need for effective conservation measures to safeguard both forest ecosystems and sustainable agricultural practices.

A Spatiotemporal Neural Network Model for Estimated-Time-of-Arrival Prediction of Flights in a Terminal Maneuvering Area

Affected by the nondeterministic nature of flight trajectories, the external environment, and airport operational conditions, the prediction of the estimated time of arrival (ETA) is one of the most challenging tasks for air traffic control in the terminal maneuvering area (TMA). Previous studies lack adequate utilization of the spatial and temporal behaviors embedded in continuous trajectories. We propose a novel spatiotemporal neural network model for estimating the time of arrival (STNN-ETA), which consists of three components: 1) trajectory pattern recognition, which classifies historical trajectories into several patterns/clusters; 2) trajectory prediction, which predicts a target flight’s subsequent positions based on trajectory pattern matching; and 3) arrival time prediction, in which nonlinear function and recurrent units are adopted to capture spatiotemporal features for prediction purposes. In the proposed model, we also utilize spatial and temporal attention mechanisms to focus on important features from radar echo maps and trajectory series, respectively, and suppress unnecessary ones for ETA prediction. To validate the effectiveness of the proposed method, we apply it to predict the ETA of flights within the Beijing TMA. Extensive experiments show that STNN-ETA outperforms the state-of-the-art models in terms of the mean absolute error.

Spatiotemporal neural network for estimating surface NO2 concentrations over north China and their human health impact

Atmospheric nitrogen dioxide (NO2) is an important reactive gas pollutant harmful to human health. The spatiotemporal coverage provided by traditional NO2 monitoring methods is insufficient, especially in the suburban and rural areas of north China, which have a high population density and experience severe air pollution. In this study, we implemented a spatiotemporal neural network (STNN) model to estimate surface NO2 from multiple sources of information, which included satellite and in situ measurements as well as meteorological and geographical data. The STNN predicted NO2 with high accuracy, with a coefficient of determination (R2) of 0.89 and a root mean squared error of 5.8 μg/m3 for sample-based 10-fold cross-validation. Based on the surface NO2 concentration determined by the STNN, we analyzed the spatial distribution and temporal trends of NO2 pollution in north China. We found substantial drops in surface NO2 concentrations ranging between 9.1% and 33.2% for large cities during the 2020 COVID-19 lockdown when compared to those in 2019. Moreover, we estimated the all-cause deaths attributed to NO2 exposure at a high spatial resolution of about 1 km, with totals of 6082, 4200, and 18,210 for Beijing, Tianjin, and Hebei Provinces in 2020, respectively. We observed remarkable regional differences in the health impacts due to NO2 among urban, suburban, and rural areas. Generally, the STNN model could incorporate spatiotemporal neighboring information and infer surface NO2 concentration with full coverage and high accuracy. Compared with machine learning regression techniques, STNN can effectively avoid model overfitting and simultaneously consider both spatial and temporal correlations of input variables using deep convolutional networks with residual blocks. The use of the proposed STNN model, as well as the surface NO2 dataset, can benefit air quality monitoring, forecasting, and health burden assessments.

Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction

The prediction of crowd flows is an important urban computing issue whose purpose is to predict the future number of incoming and outgoing people in regions. Measuring the complicated spatial–temporal dependencies with external factors, such as weather conditions and surrounding point-of-interest (POI) distribution is the most difficult aspect of predicting crowd flows movement. To overcome the above issue, this paper advises a unified dynamic deep spatio-temporal neural network model based on convolutional neural networks and long short-term memory, termed as (DHSTNet) to simultaneously predict crowd flows in every region of a city. The DHSTNet model is made up of four separate components: a recent, daily, weekly, and an external branch component. Our proposed approach simultaneously assigns various weights to different branches and integrates the four properties’ outputs to generate final predictions. Moreover, to verify the generalization and scalability of the proposed model, we apply a Graph Convolutional Network (GCN) based on Long Short Term Memory (LSTM) with the previously published model, termed as GCN-DHSTNet; to capture the spatial patterns and short-term temporal features; and to illustrate its exceptional accomplishment in predicting the traffic crowd flows. The GCN-DHSTNet model not only depicts the spatio-temporal dependencies but also reveals the influence of different time granularity, which are recent, daily, weekly periodicity and external properties, respectively. Finally, a fully connected neural network is utilized to fuse the spatio-temporal features and external properties together. Using two different real-world traffic datasets, our evaluation suggests that the proposed GCN-DHSTNet method is approximately 7.9%–27.2% and 11.2%–11.9% better than the AAtt-DHSTNet method in terms of RMSE and MAPE metrics, respectively. Furthermore, AAtt-DHSTNet outperforms other state-of-the-art methods.

Clinically localized seizure focus maybe not exactly the position of abating seizures: a computational evidence

By modeling the brain as a network, the challenge of abating seizure can be recast as a problem of network control. In the premise of bringing network under control, the minimum number of nodes prerequisite for controlling seizures are thus a natural aim of interest, which is still an outstanding issue. Here, we use the network structural control theory to guide the selection for the optimal control nodes with the aim of fully abating seizures. Firstly, we construct the dynamical complex network of pathological seizure by estimating the synchronicity and directionality of information flows over time between EEG signals from 10 patients with focal epilepsy. Then, based on the controllability and observability principles of complex systems, the minimum key nodes which are effective to fully control the network seizure behaviors are obtained. Results show that the calculated control nodes are distinct with the focus zones from clinic report. This suggests that the full control of epileptic network may not only related to the focus zone, the other non-focus nodes could also play important roles. This finding is validated by using the spatiotemporal neural network model connected with our modeled dynamical adjacent matrix. It successfully reproduces the original EEG signals which can be effectively abated by applying pulse stimulation on the identified key nodes or resecting them, while the partial effects can be obtained when functioning onto the clinically identified focus zones of 60% patients. Interestingly, for another 30% patients lesser nodes than clinic reports are need to fully control seizures. In addition, our work facilitates to identify the evolution paths of information flows, so the non-clinic focus zones identified through controllability principle can be supposed to be the potential seizure foci. In sum, our work propose a general methodology or strategy for seizures focus localizations that could comprehensively consider the time-evolving information flow and the analysis of controllability mechanisms driven by the real seizures data, as well as the computational validation. This may promote to develop the canonical computational framework with the core intent of providing support for clinical treatment decisions.

A Spatiotemporal Neural Network Modeling Method for Nonlinear Distributed Parameter Systems

Neural network (NN) has been widely used in the field of modeling of lumped parameter systems. However, an NN approach cannot be used to model complex nonlinear distributed parameter systems (DPSs) because it does not account for this type of system's relationship with space. In this article, we propose a novel spatiotemporal NN (SNN) method to model nonlinear DPSs, which considers not only nonlinear dynamics regarding time, but also a nonlinear relationship with space. A temporal NN model was first constructed to represent the nonlinear temporal dynamics of each sensor's position. A spatial distribution function was then developed to represent the nonlinear relationship between spatial points. This strategy results in inherent consideration of any spatial dynamics. Finally, by integrating both the temporal NN model and the spatial distribution function, a novel SNN model was created to represent the spatiotemporal dynamics of the nonlinear DPSs. A two-step solving approach was further developed to learn the model. Additional analysis and proof of concept showed the effectiveness of this proposed method. This proposed method is different from traditional data-driven modeling methods in that it uses full information from all sensors and does not require model reduction technology. Case studies not only demonstrate the effectiveness of this proposed method, but also its superior modeling performance as compared with several commonly used methods.

Predicting Citywide Road Traffic Flow Using Deep Spatiotemporal Neural Networks

Traffic flow forecasting has been a long-standing topic in intelligent transportation systems, and a renewed interest has been seen in recent years due to the development of artificial intelligence techniques. New deep neural networks have been developed to model traffic flow, but it is very challenging to predict citywide traffic flow at the road level in fine temporal scale owing to the influence of spatiotemporal dependencies and spatial sparsity. In this study, based on an in-depth analysis of traffic flow patterns, we propose a deep learning based spatiotemporal neural network model to predict citywide traffic flow for each road segment with high accuracy. Firstly, we examine the transformation of road network into its compact 2D image, where road segments correspond to pixels and their topological relationships are maintained in a large extent. Then, an end-to-end deep learning structure is designed to model traffic flow patterns. Specifically, recurrent convolutional network is employed to learn temporal dependencies and densely connected convolutional network is adopted to learn spatial dependencies and handle spatial sparsity. Our model attempts to aggregate the outputs of those hybrid networks using different weights, which is further enhanced by external information such as day of week. Experiments were conducted in Wuhan, China, where taxicab trajectory data were used to train and validate our model. When compared to current state of the art models, our model achieves higher accuracy in both single and multi-step traffic flow prediction tasks.

Spatiotemporal neural networks for action recognition based on joint loss

Action recognition is a challenging and important problem in a myriad of significant fields, such as intelligent robots and video surveillance. In recent years, deep learning and neural network techniques have been widely applied to action recognition and attained remarkable results. However, it is still a difficult task to recognize actions in complicated scenes, such as various illumination conditions, similar motions, and background noise. In this paper, we present a spatiotemporal neural network model with a joint loss to recognize human actions from videos. This spatiotemporal neural network is comprised of two key connected substructures. The first one is a two-stream-based network extracting optical flow and appearance features from each frame of videos, which characterizes the human actions of videos in spatial dimension. The second substructure is a group of Long Short-Term Memory structures following the spatial network, which describes the temporal and transition information in videos. This research effort presents a joint loss function for training the spatiotemporal neural network model. By introducing the loss function, the action recognition performance is improved. The proposed method was tested with video samples from two challenging datasets. The experiments demonstrate that our approach outperforms the baseline comparison methods.

A learning algorithm for saccade model with distributed feedback mechanism

In this paper we propose a new learning rule for a spatiotemporal neural network model of the primate saccadic system with a distributed feedback control mechanism. In our model the superior colliculus is represented as the distributed network model and it provides a dynamic control signal to a lumped brain stem model (Robinson–Gisbergen model). Distributed feedforward and feedback weights between the deeper layer of the superior colliculus model and the brain stem model are trained using a finite time interval learning rule based on a steepest descent method. Simulations are carried out on a 20-cell model for horizontal saccades using eye position feedback and velocity feedback, respectively. The model makes accurate saccades to all target locations over the range 2 to 15 degrees even if disturbance is added to the burst generator in the brain stem model. © 1999 Scripta Technica, Electr Eng Jpn, 129(4): 66–76, 1999.

分散型フィードバック機構を持つサッケード神経回路モデルと最適パラメータ獲得学習法

In this paper we propose a new learning rule for a spatiotemporal neural network model of the primate saccadic system with a distributed feedback control mechanism. In our model the superior colliculus is represented as the distributed network model and it provides a dynamic control signal to a lumped brain stem model (Robinson–Gisbergen model). Distributed feedforward and feedback weights between the deeper layer of the superior colliculus model and the brain stem model are trained using a finite time interval learning rule based on a steepest descent method. Simulations are carried out on a 20-cell model for horizontal saccades using eye position feedback and velocity feedback, respectively. The model makes accurate saccades to all target locations over the range 2 to 15 degrees even if disturbance is added to the burst generator in the brain stem model. © 1999 Scripta Technica, Electr Eng Jpn, 129(4): 66–76, 1999.

サッケード眼球運動の神経回路モデル

We have modified a spatio-temporal neural network model of the superior colliculus proposed by the authors. Noise was added to the superior colliculus neurons, and an eye position or velocity feedback signal was used for saccade eye movement control. Also we hypothesize that the output of the superior colliculus inhibits the pause neuron in the brainstem in order to generate saccade eye movements. In simulations our modified model succeeded in replicating accurate saccades of a variety of sizes. Simulation results and the model's relation to neurophysiological findings were discussed.

A quantitative analysis of generation of saccadic eye movements by burst neurons

A quantitative analysis of generation of saccadic eye movements by burst neurons