Spatio-temporal Neural Network Model Research Articles

Spatio-temporal neural networks for monitoring and prediction of CO[formula omitted] plume migration from measurable field data

Carbon capture and storage (CCS) technologies are crucial for mitigating greenhouse gas emissions. The success of CCS projects hinges on accurately predicting and monitoring the CO2 plume migration during and after injection. To address the computational burden of traditional numerical simulation methods, previous studies have used neural networks as proxy models to expedite the prediction of the CO2 plume migration. However, these models rely on uncertain inputs, such as the distribution of heterogeneous rock permeability and porosity maps, which can lead to erroneous predictions and limit their adoption in real-world applications. To address this issue, this study introduces a framework for reconstruction and short-term prediction of CO2 plume migration based solely on field measurements that directly provide information on the migration of the CO2 plumes, thus eliminating the dependency on uncertain geological information as model input. The framework trains a spatio-temporal neural network model using simulated data under various geologic settings to capture the plume evolution dynamics without constraining it to a specific geological scenario. Once trained, the model integrates global and local field measurements from multiple sources to reconstruct the CO2 plume and predict its spatio-temporal evolution. The effectiveness of the proposed framework is tested using two case studies: one using a synthetic dataset and another with data generated from a model of a real field in the Southern San Joaquin Basin. The results show that the proposed framework can accurately reconstruct and perform short-term predictions of the CO2 plume migration by integrating various forms of field measurements.

A study on the efficacy of an attention mechanism-based strategy planning method for stock trading portfolios

In order to take full advantage of the fact that there is a large amount of information and data available in the field of stock trading, this paper develops a new type of stock trading strategy, which reduces investment risk. A stock trading portfolio strategy planning method based on an attention mechanism, as disclosed in the present invention, relates to the field of stock trading technology. The introduction of an attention mechanism allows the strategy to focus on those stocks with higher evaluation coefficients, thereby improving the selectivity and accuracy of the strategy and thus the effectiveness of the portfolio strategy. Furthermore, based on the prediction results of the deep spatio-temporal neural network model, it is possible to determine the stock's short-term trends and identify the optimal trading points. This information can be utilized as a reference for investors, assisting them in making more informed decisions regarding the timing of buying and selling, improving the success rate and yield of trading, enabling them to swiftly and accurately assess the performance of each stock, facilitating timely adjustments to their portfolios, and enabling them to devise the optimal trading strategy and timing for each stock, enhancing the accuracy and efficiency of their trading decisions.

Forecasting sea surface temperature during typhoon events in the Bohai Sea using spatiotemporal neural networks

Advanced neural network methods can effectively combine temporal and spatial information, allowing us to exploit complex relationships in the prediction of sea surface temperature (SST). Therefore, the authors have developed an attention-based context fusion network model recently, known as ACFN, to enhance the underlying spatiotemporal correlations in SST data. In a previous preliminary long-term evaluation, ACFN has demonstrated substantial improvements over several state-of-the-art baseline models, such as Convolutional Long-Short Term Memory (ConvLSTM) and Predictive Recurrent Neural Network (PredRNN). However, these methods have not been widely used for predicting SST during typhoons, and their performance has not been thoroughly evaluated. This study focuses on three specific typhoons, In-fa (2021), Lekima (2019), and Rumbia (2018), as case studies. The results show that ACFN consistently outperforms ConvLSTM and PredRNN, as demonstrated during Typhoon In-fa with a mean absolute error of 0.27 °C and a root-mean-square error of 0.33 °C for a 1-day forecast time. This study also examines the impact of forecast time on predictive skill, revealing that the performance of ACFN, as expected, gradually decreases with longer forecast times. This thorough evaluation of the ACFN model provides a valuable reference for using deep learning in predicting regional typhoon SST. Plain language summaryIn previous work, we developed an attention-based context fusion network model. This spatiotemporal neural network model has been successfully applied to short-term sea surface temperature (SST) prediction in the Bohai Sea, utilizing only current and previous 1- to 10-day SST data as input to forecast SST for the subsequent 1- to 10-day period. In this study, we further evaluated the capability of the model to predict SST during typhoons in the Bohai Sea, and the results were very promising. When predicting SST during Typhoon In-fa, the present spatiotemporal neural network model had a mean absolute error of only 0.27 °C for a 1-day forecast time, and a correlation coefficient of 0.74. On the other hand, comparison with baseline spatiotemporal neural network models shows that the present model is more effective in extracting input information. The current model has resolved the constraint that its predictive capability does not diminish as the forecast time increases, thereby enhancing the rationality of the prediction outcomes. In conclusion, the present spatiotemporal neural network model demonstrates its efficacy in forecasting typhoon-induced SST in the Bohai Sea, offering valuable insights for applications in meteorology and oceanography.

Long-term drought prediction using deep neural networks based on geospatial weather data

The problem of high-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance. Yet, it is still unsolved with reasonable accuracy due to data complexity and aridity stochasticity. We tackle drought data by introducing an end-to-end approach that adopts a spatio-temporal neural network model with accessible open monthly climate data as the input. Our systematic research employs diverse proposed models and five distinct environmental regions as a testbed to evaluate the efficacy of the Palmer Drought Severity Index (PDSI) prediction. Key aggregated findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts. At the same time, the Convolutional LSTM excels in longer-term forecasting. Both models achieved high ROC AUC scores: 0.948 for one month ahead and 0.617 for twelve months ahead forecasts, becoming closer to perfect ROC-AUC by 54% and 16%, respectively, c.t. classic approaches.

A macro–micro spatio-temporal neural network for traffic prediction

Accurate traffic prediction is crucial for planning, management and control of intelligent transportation systems. Most state-of-the-art methods for traffic prediction effectively capture complex traffic patterns (e.g. spatial and temporal correlations of traffic data) by employing spatio-temporal neural networks as prediction models, together with graph convolution networks to learn spatial correlations of prediction objects (e.g. traffic states of road segments, as in this study). Such spatial correlations can be regarded as micro correlations. However, there are also macro correlations between regions, each of which is composed of multiple road segments or artificially partitioned areas. Macro correlations represent another type of interaction within road segments, and should be carefully considered when predicting traffic. The diversity of micro spatial correlations and corresponding macro spatial correlations (e.g. correlations based on physical proximity or traffic pattern similarity) further increases the complexity of traffic prediction. We overcome these challenges by developing a macro–micro spatio-temporal neural network model, denoted ‘MMSTNet’. MMSTNet captures spatio-temporal patterns by (a) utilizing a graph convolution network and a spatial attention network to capture micro and macro spatial correlations, respectively; (b) employing a temporal convolution network and a temporal attention network to learn temporal patterns; and (c) integrating hierarchically learned representations based on designed attention mechanisms. We perform evaluations on two real-world datasets and thereby demonstrate that MMSTNet outperforms state-of-the-art models in traffic prediction tasks.

Assessing the impacts of past and ongoing deforestation on rainfall patterns in South America.

Despite recent advances in modeling forest-rainfall relationships, the current understanding of changes in observed rainfall patterns resulting from historical deforestation remains limited. To address this knowledge gap, we analyzed how 40 years of deforestation has altered rainfall patterns in South America as well as how current Amazonian forest cover sustains rainfall. First, we develop a spatiotemporal neural network model to simulate rainfall as a function of vegetation and climate inputs in South America; second, we assess the rainfall effects of observed deforestation in South America during the periods 1982-2020 and 2000-2020; third, we assess the potential rainfall changes in the Amazon biome under two deforestation scenarios. We find that, on average, cumulative deforestation in South America from 1982 to 2020 has reduced rainfall over the period 2016-2020 by 18% over deforested areas, and by 9% over non-deforested areas across South America. We also find that more recent deforestation, that is, from 2000 to 2020, has reduced rainfall over the period 2016-2020 by 10% over deforested areas and by 5% over non-deforested areas. Deforestation between 1982 and 2020 has led to a doubling in the area experiencing a minimum dry season of 4 months in the Amazon biome. Similarly, in the Cerrado region, there has been a corresponding doubling in the area with a minimum dry season of 7 months. These changes are compared to a hypothetical scenario where no deforestation occurred. Complete conversion of all Amazon forest land outside protected areas would reduce average annual rainfall in the Amazon by 36% and complete deforestation of all forest cover including protected areas would reduce average annual rainfall in the Amazon by 68%. Our findings emphasize the urgent need for effective conservation measures to safeguard both forest ecosystems and sustainable agricultural practices.

A Spatiotemporal Neural Network Model for Estimated-Time-of-Arrival Prediction of Flights in a Terminal Maneuvering Area

Affected by the nondeterministic nature of flight trajectories, the external environment, and airport operational conditions, the prediction of the estimated time of arrival (ETA) is one of the most challenging tasks for air traffic control in the terminal maneuvering area (TMA). Previous studies lack adequate utilization of the spatial and temporal behaviors embedded in continuous trajectories. We propose a novel spatiotemporal neural network model for estimating the time of arrival (STNN-ETA), which consists of three components: 1) trajectory pattern recognition, which classifies historical trajectories into several patterns/clusters; 2) trajectory prediction, which predicts a target flight’s subsequent positions based on trajectory pattern matching; and 3) arrival time prediction, in which nonlinear function and recurrent units are adopted to capture spatiotemporal features for prediction purposes. In the proposed model, we also utilize spatial and temporal attention mechanisms to focus on important features from radar echo maps and trajectory series, respectively, and suppress unnecessary ones for ETA prediction. To validate the effectiveness of the proposed method, we apply it to predict the ETA of flights within the Beijing TMA. Extensive experiments show that STNN-ETA outperforms the state-of-the-art models in terms of the mean absolute error.

Spatiotemporal neural network for estimating surface NO2 concentrations over north China and their human health impact

Atmospheric nitrogen dioxide (NO2) is an important reactive gas pollutant harmful to human health. The spatiotemporal coverage provided by traditional NO2 monitoring methods is insufficient, especially in the suburban and rural areas of north China, which have a high population density and experience severe air pollution. In this study, we implemented a spatiotemporal neural network (STNN) model to estimate surface NO2 from multiple sources of information, which included satellite and in situ measurements as well as meteorological and geographical data. The STNN predicted NO2 with high accuracy, with a coefficient of determination (R2) of 0.89 and a root mean squared error of 5.8 μg/m3 for sample-based 10-fold cross-validation. Based on the surface NO2 concentration determined by the STNN, we analyzed the spatial distribution and temporal trends of NO2 pollution in north China. We found substantial drops in surface NO2 concentrations ranging between 9.1% and 33.2% for large cities during the 2020 COVID-19 lockdown when compared to those in 2019. Moreover, we estimated the all-cause deaths attributed to NO2 exposure at a high spatial resolution of about 1 km, with totals of 6082, 4200, and 18,210 for Beijing, Tianjin, and Hebei Provinces in 2020, respectively. We observed remarkable regional differences in the health impacts due to NO2 among urban, suburban, and rural areas. Generally, the STNN model could incorporate spatiotemporal neighboring information and infer surface NO2 concentration with full coverage and high accuracy. Compared with machine learning regression techniques, STNN can effectively avoid model overfitting and simultaneously consider both spatial and temporal correlations of input variables using deep convolutional networks with residual blocks. The use of the proposed STNN model, as well as the surface NO2 dataset, can benefit air quality monitoring, forecasting, and health burden assessments.

Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction

The prediction of crowd flows is an important urban computing issue whose purpose is to predict the future number of incoming and outgoing people in regions. Measuring the complicated spatial–temporal dependencies with external factors, such as weather conditions and surrounding point-of-interest (POI) distribution is the most difficult aspect of predicting crowd flows movement. To overcome the above issue, this paper advises a unified dynamic deep spatio-temporal neural network model based on convolutional neural networks and long short-term memory, termed as (DHSTNet) to simultaneously predict crowd flows in every region of a city. The DHSTNet model is made up of four separate components: a recent, daily, weekly, and an external branch component. Our proposed approach simultaneously assigns various weights to different branches and integrates the four properties’ outputs to generate final predictions. Moreover, to verify the generalization and scalability of the proposed model, we apply a Graph Convolutional Network (GCN) based on Long Short Term Memory (LSTM) with the previously published model, termed as GCN-DHSTNet; to capture the spatial patterns and short-term temporal features; and to illustrate its exceptional accomplishment in predicting the traffic crowd flows. The GCN-DHSTNet model not only depicts the spatio-temporal dependencies but also reveals the influence of different time granularity, which are recent, daily, weekly periodicity and external properties, respectively. Finally, a fully connected neural network is utilized to fuse the spatio-temporal features and external properties together. Using two different real-world traffic datasets, our evaluation suggests that the proposed GCN-DHSTNet method is approximately 7.9%–27.2% and 11.2%–11.9% better than the AAtt-DHSTNet method in terms of RMSE and MAPE metrics, respectively. Furthermore, AAtt-DHSTNet outperforms other state-of-the-art methods.

Clinically localized seizure focus maybe not exactly the position of abating seizures: a computational evidence

By modeling the brain as a network, the challenge of abating seizure can be recast as a problem of network control. In the premise of bringing network under control, the minimum number of nodes prerequisite for controlling seizures are thus a natural aim of interest, which is still an outstanding issue. Here, we use the network structural control theory to guide the selection for the optimal control nodes with the aim of fully abating seizures. Firstly, we construct the dynamical complex network of pathological seizure by estimating the synchronicity and directionality of information flows over time between EEG signals from 10 patients with focal epilepsy. Then, based on the controllability and observability principles of complex systems, the minimum key nodes which are effective to fully control the network seizure behaviors are obtained. Results show that the calculated control nodes are distinct with the focus zones from clinic report. This suggests that the full control of epileptic network may not only related to the focus zone, the other non-focus nodes could also play important roles. This finding is validated by using the spatiotemporal neural network model connected with our modeled dynamical adjacent matrix. It successfully reproduces the original EEG signals which can be effectively abated by applying pulse stimulation on the identified key nodes or resecting them, while the partial effects can be obtained when functioning onto the clinically identified focus zones of 60% patients. Interestingly, for another 30% patients lesser nodes than clinic reports are need to fully control seizures. In addition, our work facilitates to identify the evolution paths of information flows, so the non-clinic focus zones identified through controllability principle can be supposed to be the potential seizure foci. In sum, our work propose a general methodology or strategy for seizures focus localizations that could comprehensively consider the time-evolving information flow and the analysis of controllability mechanisms driven by the real seizures data, as well as the computational validation. This may promote to develop the canonical computational framework with the core intent of providing support for clinical treatment decisions.

A Spatiotemporal Neural Network Modeling Method for Nonlinear Distributed Parameter Systems

Neural network (NN) has been widely used in the field of modeling of lumped parameter systems. However, an NN approach cannot be used to model complex nonlinear distributed parameter systems (DPSs) because it does not account for this type of system's relationship with space. In this article, we propose a novel spatiotemporal NN (SNN) method to model nonlinear DPSs, which considers not only nonlinear dynamics regarding time, but also a nonlinear relationship with space. A temporal NN model was first constructed to represent the nonlinear temporal dynamics of each sensor's position. A spatial distribution function was then developed to represent the nonlinear relationship between spatial points. This strategy results in inherent consideration of any spatial dynamics. Finally, by integrating both the temporal NN model and the spatial distribution function, a novel SNN model was created to represent the spatiotemporal dynamics of the nonlinear DPSs. A two-step solving approach was further developed to learn the model. Additional analysis and proof of concept showed the effectiveness of this proposed method. This proposed method is different from traditional data-driven modeling methods in that it uses full information from all sensors and does not require model reduction technology. Case studies not only demonstrate the effectiveness of this proposed method, but also its superior modeling performance as compared with several commonly used methods.

Predicting Citywide Road Traffic Flow Using Deep Spatiotemporal Neural Networks

Traffic flow forecasting has been a long-standing topic in intelligent transportation systems, and a renewed interest has been seen in recent years due to the development of artificial intelligence techniques. New deep neural networks have been developed to model traffic flow, but it is very challenging to predict citywide traffic flow at the road level in fine temporal scale owing to the influence of spatiotemporal dependencies and spatial sparsity. In this study, based on an in-depth analysis of traffic flow patterns, we propose a deep learning based spatiotemporal neural network model to predict citywide traffic flow for each road segment with high accuracy. Firstly, we examine the transformation of road network into its compact 2D image, where road segments correspond to pixels and their topological relationships are maintained in a large extent. Then, an end-to-end deep learning structure is designed to model traffic flow patterns. Specifically, recurrent convolutional network is employed to learn temporal dependencies and densely connected convolutional network is adopted to learn spatial dependencies and handle spatial sparsity. Our model attempts to aggregate the outputs of those hybrid networks using different weights, which is further enhanced by external information such as day of week. Experiments were conducted in Wuhan, China, where taxicab trajectory data were used to train and validate our model. When compared to current state of the art models, our model achieves higher accuracy in both single and multi-step traffic flow prediction tasks.

Spatiotemporal neural networks for action recognition based on joint loss

Action recognition is a challenging and important problem in a myriad of significant fields, such as intelligent robots and video surveillance. In recent years, deep learning and neural network techniques have been widely applied to action recognition and attained remarkable results. However, it is still a difficult task to recognize actions in complicated scenes, such as various illumination conditions, similar motions, and background noise. In this paper, we present a spatiotemporal neural network model with a joint loss to recognize human actions from videos. This spatiotemporal neural network is comprised of two key connected substructures. The first one is a two-stream-based network extracting optical flow and appearance features from each frame of videos, which characterizes the human actions of videos in spatial dimension. The second substructure is a group of Long Short-Term Memory structures following the spatial network, which describes the temporal and transition information in videos. This research effort presents a joint loss function for training the spatiotemporal neural network model. By introducing the loss function, the action recognition performance is improved. The proposed method was tested with video samples from two challenging datasets. The experiments demonstrate that our approach outperforms the baseline comparison methods.

A learning algorithm for saccade model with distributed feedback mechanism

In this paper we propose a new learning rule for a spatiotemporal neural network model of the primate saccadic system with a distributed feedback control mechanism. In our model the superior colliculus is represented as the distributed network model and it provides a dynamic control signal to a lumped brain stem model (Robinson–Gisbergen model). Distributed feedforward and feedback weights between the deeper layer of the superior colliculus model and the brain stem model are trained using a finite time interval learning rule based on a steepest descent method. Simulations are carried out on a 20-cell model for horizontal saccades using eye position feedback and velocity feedback, respectively. The model makes accurate saccades to all target locations over the range 2 to 15 degrees even if disturbance is added to the burst generator in the brain stem model. © 1999 Scripta Technica, Electr Eng Jpn, 129(4): 66–76, 1999.

分散型フィードバック機構を持つサッケード神経回路モデルと最適パラメータ獲得学習法

In this paper we propose a new learning rule for a spatiotemporal neural network model of the primate saccadic system with a distributed feedback control mechanism. In our model the superior colliculus is represented as the distributed network model and it provides a dynamic control signal to a lumped brain stem model (Robinson–Gisbergen model). Distributed feedforward and feedback weights between the deeper layer of the superior colliculus model and the brain stem model are trained using a finite time interval learning rule based on a steepest descent method. Simulations are carried out on a 20-cell model for horizontal saccades using eye position feedback and velocity feedback, respectively. The model makes accurate saccades to all target locations over the range 2 to 15 degrees even if disturbance is added to the burst generator in the brain stem model. © 1999 Scripta Technica, Electr Eng Jpn, 129(4): 66–76, 1999.

サッケード眼球運動の神経回路モデル

We have modified a spatio-temporal neural network model of the superior colliculus proposed by the authors. Noise was added to the superior colliculus neurons, and an eye position or velocity feedback signal was used for saccade eye movement control. Also we hypothesize that the output of the superior colliculus inhibits the pause neuron in the brainstem in order to generate saccade eye movements. In simulations our modified model succeeded in replicating accurate saccades of a variety of sizes. Simulation results and the model's relation to neurophysiological findings were discussed.

A quantitative analysis of generation of saccadic eye movements by burst neurons

A quantitative analysis of generation of saccadic eye movements by burst neurons