Spatio-temporal Neural Network Research Articles

Distributionally robust hospital capacity expansion planning under stochastic and correlated patient demand

This paper investigates the optimal locations and capacities of hospital expansion facilities under uncertain future patient demands, considering both spatial and temporal correlations. We propose a novel two-stage distributionally robust optimization (DRO) model that integrates a Spatio-Temporal Neural Network (STNN). Specifically, we develop an STNN model that predicts future hospital occupancy levels considering spatial and temporal patterns in time-series datasets over a network of hospitals. The predictions of the STNN model are then used in the construction of the ambiguity set of the DRO model. To address computational challenges associated with two-stage DRO, we employ the linear-decision-rules technique to derive a tractable mixed-integer linear programming approximation. Extensive computational experiments conducted on real-world data demonstrate the superiority of the STNN model in minimizing forecast errors. Compared to neural network models built for each individual hospital, the proposed STNN model achieves a 53% improvement in average root mean square error. Furthermore, the results demonstrate the value of incorporating spatiotemporal dependencies of demand uncertainty in the DRO model, as evidenced by out-of-sample analysis conducted with both ground truth data and under perfect information scenarios.

Finite-time synchronization analysis for the generalized Caputo fractional spatio-temporal neural networks

Finite-time synchronization analysis for the generalized Caputo fractional spatio-temporal neural networks

Spatio-temporal neural networks for monitoring and prediction of CO[formula omitted] plume migration from measurable field data

Carbon capture, utilization, and storage (CCUS) technologies are crucial for mitigating greenhouse gas emissions. The success of CCUS projects hinges on accurate prediction and monitoring of the CO2 plume migration during and after injection. To address the computational burden of traditional numerical simulation methods, previous studies have successfully used neural networks as proxy models to expedite the prediction of the CO2 plume migration. However, these models rely on uncertain inputs, such as the distribution of heterogeneous permeability and porosity maps, which can lead to erroneous predictions and pose a significant hurdle for their adoption in real-world applications. To address this issue, this study introduces a framework for reconstruction and short-term prediction of CO2 plume migration based solely on field measurements, which can directly provide information on CO2 plumes, thus eliminating the dependency on uncertain geological information as model input. The framework trains a spatio-temporal neural network model using simulated data under various geologic scenarios to capture the plume evolution dynamics without constraining it to specific geological scenarios. Once trained, the model integrates global and local field measurements from multiple sources to reconstruct the CO2 plume and predict its spatio-temporal evolution. The effectiveness of the proposed framework is tested using two case studies: one using a synthetic dataset and another with data generated from a model of a real field in the Southern San Joaquin Basin. The results show that the proposed framework can accurately reconstruct and perform short-term predictions of the CO2 plume migration by integrating various forms of field measurements.

A multi-task spatio-temporal fully convolutional model incorporating interaction patterns for traffic flow prediction

Previous traffic flow prediction studies have utilized spatio-temporal neural networks combined with the multi-task learning framework to seek complementary information for enhancing prediction performance. However, the existing methods still face two challenges: they fail to capture global interaction patterns between regions and lack consideration for inter-correlations within interaction patterns. To solve these issues, we propose a novel multi-task spatio-temporal fully convolutional model named MSTFCM. First, the model includes the interaction tensor and raster tensor as task inputs, where the interaction tensor extends the raster tensor by incorporating global interaction patterns between regions. Second, a multi-task framework combined spatio-temporal convolutional block was used to learn generalized features and interaction features. A channel spatio-temporal attention is added to adaptively adjust feature weights and capture inter-correlations. To train the MSTFCM, the uncertainty loss was designed as the learnable loss functions, which capture various flow fluctuations, to facilitate multi-task optimization. The proposed model was validated on two real-world traffic datasets collected in Xiamen, China. Experimental results showed that MSTFCM outperformed nine baselines in one-step and multi-step prediction, with slower performance degradation as predicted time intervals and steps increased. We further validated the model’s effectiveness through designed variants and visualization results.

Spatiotemporal weighted neural network reveals surface seawater pCO2 distributions and underlying environmental mechanisms in the North Pacific Ocean

The North Pacific Ocean plays a pivotal role as a carbon sink within the global carbon cycle. However, a comprehensive understanding of the spatiotemporal dynamics of carbon dioxide concentration and its determinants in this domain remains elusive due to its vast dimensions and the intricacies of influencing factors, with previous research on carbon dioxide partial pressure in the North Pacific Ocean also being relatively scarce. While prevalent machine learning methodologies have been extensively applied to predict the partial pressure of ocean carbon dioxide (pCO2), their limited interpretability has impeded substantial progress in elucidating the underlying mechanisms. This study introduces a gridded spatiotemporal neural network weighted regression (GSTNNWR) model to illuminate temporal and spatial relationships among relevant environmental variables and pCO2. The GSTNNWR model achieves high-precision and high-resolution forecasts of surface pCO2 in the North Pacific Ocean, demonstrating commendable performance (R2 = 0.863 and RMSE=15.123 µatm). Simultaneously, we obtain a quantitative characterization of how various environmental factors influence pCO2 across different temporal and spatial scales. Results show a dominant positive effect of temperature on the pCO2, with an averaged normalized coefficient of 0.28, and variability in the effects of chlorophyll and salinity on the pCO2 at different spatial and temporal locations and temperatures, whose average normalized coefficients are −0.10 and −0.04.The findings of our study will provide insights into the mechanisms and interactions within the North Pacific carbon cycle, contributing to a better understanding of ocean carbon sink formation and the dynamic regulation of the North Pacific carbon cycle.

Image-driven prediction of fatigue crack growth in metal materials via spatiotemporal neural network

This study proposes an image-driven model based on the SimVP spatiotemporal neural network (STNN) to predict the fatigue crack growth (FCG) in aluminum alloys. This methodology represents a novel usage of STNNs for FCG analysis. It does not require repetitive modeling, extensive computations, or conventional mechanical assumptions. The datasets used during this study were gathered from fatigue experiments with a variety of crack positions, angles, and load levels; they contained a total of 17,925 image frames obtained from DIC measurements. Subsequently, the displacement fields were interpolated onto uniform grids and then augmented, so they could be fitted into an STNN. The proposed method was validated using specimens with edge and central cracks subjected to loads equal to 15.0 % and 20.0 % of the ultimate load. The generalization capability of the proposed method was studied by predicting the FCG under load levels and crack angles outside the training set. In addition, its predictive capability was investigated for both short and long step sizes by employing datasets in which the image data were collected at varying intervals. The overall structural similarity index measurement values were greater than 0.968, and the root mean square errors were held within 0.025 mm. The predicted displacement fields, crack lengths, and crack growth rates agreed well with experimental measurements.

A study on the efficacy of an attention mechanism-based strategy planning method for stock trading portfolios

In order to take full advantage of the fact that there is a large amount of information and data available in the field of stock trading, this paper develops a new type of stock trading strategy, which reduces investment risk. A stock trading portfolio strategy planning method based on an attention mechanism, as disclosed in the present invention, relates to the field of stock trading technology. The introduction of an attention mechanism allows the strategy to focus on those stocks with higher evaluation coefficients, thereby improving the selectivity and accuracy of the strategy and thus the effectiveness of the portfolio strategy. Furthermore, based on the prediction results of the deep spatio-temporal neural network model, it is possible to determine the stock's short-term trends and identify the optimal trading points. This information can be utilized as a reference for investors, assisting them in making more informed decisions regarding the timing of buying and selling, improving the success rate and yield of trading, enabling them to swiftly and accurately assess the performance of each stock, facilitating timely adjustments to their portfolios, and enabling them to devise the optimal trading strategy and timing for each stock, enhancing the accuracy and efficiency of their trading decisions.

Spatiotemporal Neural Network for Sublexical Information Processing: An Intracranial SEEG Study.

Words offer a unique opportunity to separate the processing mechanisms of object subcomponents from those of the whole object, because the phonological or semantic information provided by the word subcomponents (i.e., sublexical information) can conflict with that provided by the whole word (i.e., lexical information). Previous studies have revealed some of the specific brain regions and temporal information involved in sublexical information processing. However, a comprehensive spatiotemporal neural network for sublexical processing remains to be fully elucidated due to the low temporal or spatial resolutions of previous neuroimaging studies. In this study, we recorded stereoelectroencephalography (SEEG) signals with high spatial and temporal resolutions from a large sample of 39 epilepsy patients (both sexes) during a Chinese character oral reading task. We explored the activated brain regions and their connectivity related to three sublexical effects: phonological regularity (whether the whole character's pronunciation aligns with its phonetic radical), phonological consistency (whether characters with the same phonetic radical share the same pronunciation), and semantic transparency (whether the whole character's meaning aligns with its semantic radical). The results revealed that sublexical effects existed in the inferior frontal gyrus, precentral and postcentral gyri, temporal lobe, and middle occipital gyrus. Additionally, connectivity from the middle occipital gyrus to the postcentral gyrus and from postcentral gyrus to the fusiform gyrus was associated with the sublexical effects. These findings provide valuable insights into the spatiotemporal dynamics of sublexical processing and object recognition in the brain.Significance statement Elucidating the intricate neural mechanisms underlying sublexical processing is crucial for understanding the intricacies of language comprehension and object recognition in the human brain. This study employed intracranial stereoelectroencephalography (SEEG) recordings to investigate the spatiotemporal dynamics of sublexical processing during a Chinese character reading task. We constructed a neural network for sublexical processing and depicted its temporal sequence in different brain regions. Furthermore, we identified the information flow within this network and observed its variation with the reading of characters containing different sublexical information. These findings not only advance our understanding of the cerebral mechanisms governing sublexical processing but also offer insights into the broader framework of object recognition processes.

Adaptive spatiotemporal neural networks through complementary hybridization

Processing spatiotemporal data sources with both high spatial dimension and rich temporal information is a ubiquitous need in machine intelligence. Recurrent neural networks in the machine learning domain and bio-inspired spiking neural networks in the neuromorphic computing domain are two promising candidate models for dealing with spatiotemporal data via extrinsic dynamics and intrinsic dynamics, respectively. Nevertheless, these networks have disparate modeling paradigms, which leads to different performance results, making it hard for them to cover diverse data sources and performance requirements in practice. Constructing a unified modeling framework that can effectively and adaptively process variable spatiotemporal data in different situations remains quite challenging. In this work, we propose hybrid spatiotemporal neural networks created by combining the recurrent neural networks and spiking neural networks under a unified surrogate gradient learning framework and a Hessian-aware neuron selection method. By flexibly tuning the ratio between two types of neurons, the hybrid model demonstrates better adaptive ability in balancing different performance metrics, including accuracy, robustness, and efficiency on several typical benchmarks, and generally outperforms conventional single-paradigm recurrent neural networks and spiking neural networks. Furthermore, we evidence the great potential of the proposed network with a robotic task in varying environments. With our proof of concept, the proposed hybrid model provides a generic modeling route to process spatiotemporal data sources in the open world.

A deep learning-based convolutional spatiotemporal network proxy model for reservoir production prediction

Accurate production prediction is crucial in the field of reservoir management and production optimization. Traditional models often struggle with the complexities of nonlinear relationships and high-dimensional data, which hinders their ability to capture the variability of the production process efficiently and results in time-consuming calculations. To overcome these limitations, this paper introduces an innovative proxy modeling technique employing a convolutional spatiotemporal neural network. This method utilizes convolutional neural networks to extract spatial features from high-dimensional data, while the Transformer is used to model and predict complex temporal dynamics in production. To validate the effectiveness of the proposed proxy model, two case studies involving four injection and nine production wells within two-dimensional (2D) and three-dimensional (3D) non-homogeneous reservoirs were conducted, with the R2 coefficient serving as the primary evaluation metric. As the number of training iterations and data volume increase, the proxy model demonstrates rapid convergence. In tests conducted on the 2D and 3D datasets, the average R2 value exceeded 0.96 and 0.94. These results confirm the accuracy and stability of the proxy model. It also shows that the proxy model can accurately describe the geological and fluid seepage characteristics of the reservoir, which in turn can achieve a highly accurate match with the real data. In addition, the computational time is reduced by two orders of magnitude compared to traditional models. Compared with the long short-term memory method, the accuracy of the prediction results is increased by 30%, which greatly enhances efficiency and accuracy. To some extent, the presented proxy model can provide some guidance for the efficient history match of production data.

Porosity prediction using a deep learning method based on bidirectional spatio-temporal neural network

Deep learning is one of the best machine learning algorithms for modeling complex mapping relationships between independent and dependent variables, and thus it can be viewed as an ideal approach to predict porosity. In this study, to overcome the deficiencies in current porosity prediction based on deep learning and improve the prediction accuracy, we proposed a deep learning model based on bidirectional temporal convolutional network (BTCN) and bidirectional long short-term memory (BLSTM) network, called bidirectional spatio-temporal neural network (BSTNN), to establish a porosity prediction model. First, the maximum information coefficient is used to analyze the correlation between well logs and porosity, which provides a basis for determining the inputs of the prediction model. Then, a hybrid network structure is constructed by using BTCN and BLSTM, in which BTCN goes to learn the bidirectional long sequence features and BLSTM goes to learn the variation trend and context information with depth, so the hybrid network structure can learn richer logging signal features. Finally, the extracted features are passed through the fully connected layer to output the porosity prediction results. Porosity prediction experiment are conducted by using the actual field data set. The results show that the proposed method has the lower prediction errors for the porosity modeling (RMSE = 0.368 and MAE = 0.260) compared to the benchmark models convolutional neural network (RMSE = 0.404 and MAE = 0.292) and long short-term memory network (RMSE = 0.418 and MAE = 0.298), which verifies the effectiveness of this prediction method.

Enhanced Air Quality Prediction through Spatio-temporal Feature Sxtraction and Fusion: A Self-tuning Hybrid Approach with GCN and GRU

Accurate prediction of air quality change is essential for air pollution control and human daily mobility. Due to the strong spatial and temporal correlation of air quality changes, existing air quality prediction methods often face the problem of low prediction accuracy due to insufficient extraction of spatio-temporal features. In this paper, we proposed a self-tuning spatio-temporal neural network (ST2NN) to enhance air quality prediction. ST2NN model consisted of four modules. First, ST2NN model constructed a temporal feature extraction module and a spatial feature extraction module based on gated recurrent unit (GRU) and graph convolutional neural network (GCN), respectively, and the two feature extraction modules adopted a parallel structure, which could effectively extract the spatio-temporal features in data. Additionally, ST2NN model constructed a feature fusion module based on gating mechanism to delineate the contribution of spatio-temporal features to the predicted values. Further, ST2NN model constructed a Hyperband hyperparameter optimization module based on Hyperband optimization algorithm to automatically adjust the network hyperparameters. The structure of ST2NN model endowed it with excellent spatio-temporal feature extraction and parameter adaptability. ST2NN model was evaluated and compared with existing models, including convolutional long short-term memory neural network (ConvLSTM), GRU, combined convolutional neural network and long short-term memory neural network (CNN-LSTM), and GCN-LSTM for air quality index (AQI) prediction using data from twelve monitoring stations in Beijing, China. Across all four evaluation indexes, ST2NN model outperformed the comparative models, improving prediction accuracy by 0.51%-10.18% (measured using R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${R}^{2}$$\\end{document}). From the experimental results, it can be seen that ST2NN model constructed from the perspective of spatio-temporal feature extraction has better prediction performance compared with the existing air quality prediction model, which provides a new method for air quality prediction and has certain application value.

Adaptive Decision Spatio-temporal neural ODE for traffic flow forecasting with Multi-Kernel Temporal Dynamic Dilation Convolution

Traffic flow prediction is crucial for efficient traffic management. It involves predicting vehicle movement patterns to reduce congestion and enhance traffic flow. However, the highly non-linear and complex patterns commonly observed in traffic flow pose significant challenges for this task. Current Graph Neural Network (GNN) models often construct shallow networks, which limits their ability to extract deeper spatio-temporal representations. Neural ordinary differential equations for traffic prediction address over-smoothing but require significant computational resources, leading to inefficiencies, and sometimes deeper networks may lead to poorer predictions for complex traffic information. In this study, we propose an Adaptive Decision spatio-temporal Neural Ordinary Differential Network, which can adaptively determine the number of layers of ODE according to the complexity of traffic information. It can solve the over-smoothing problem better, improving overall efficiency and prediction accuracy. In addition, traditional temporal convolution methods make it difficult to deal with complex and variable traffic time information with a large time span. Therefore, we introduce a multi-kernel temporal dynamic expansive convolution to handle the traffic time information. Multi-kernel temporal dynamic expansive convolution employs a dynamic dilation strategy, dynamically adjusting the network’s receptive field across levels, effectively capturing temporal dependencies, and can better adapt to the changing time data of traffic information. Additionally, multi-kernel temporal dynamic expansive convolution integrates multi-scale convolution kernels, enabling the model to learn features across diverse temporal scales. We evaluated our proposed method on several real-world traffic datasets. Experimental results show that our method outperformed state-of-the-art benchmarks.

Forecasting sea surface temperature during typhoon events in the Bohai Sea using spatiotemporal neural networks

Advanced neural network methods can effectively combine temporal and spatial information, allowing us to exploit complex relationships in the prediction of sea surface temperature (SST). Therefore, the authors have developed an attention-based context fusion network model recently, known as ACFN, to enhance the underlying spatiotemporal correlations in SST data. In a previous preliminary long-term evaluation, ACFN has demonstrated substantial improvements over several state-of-the-art baseline models, such as Convolutional Long-Short Term Memory (ConvLSTM) and Predictive Recurrent Neural Network (PredRNN). However, these methods have not been widely used for predicting SST during typhoons, and their performance has not been thoroughly evaluated. This study focuses on three specific typhoons, In-fa (2021), Lekima (2019), and Rumbia (2018), as case studies. The results show that ACFN consistently outperforms ConvLSTM and PredRNN, as demonstrated during Typhoon In-fa with a mean absolute error of 0.27 °C and a root-mean-square error of 0.33 °C for a 1-day forecast time. This study also examines the impact of forecast time on predictive skill, revealing that the performance of ACFN, as expected, gradually decreases with longer forecast times. This thorough evaluation of the ACFN model provides a valuable reference for using deep learning in predicting regional typhoon SST. Plain language summaryIn previous work, we developed an attention-based context fusion network model. This spatiotemporal neural network model has been successfully applied to short-term sea surface temperature (SST) prediction in the Bohai Sea, utilizing only current and previous 1- to 10-day SST data as input to forecast SST for the subsequent 1- to 10-day period. In this study, we further evaluated the capability of the model to predict SST during typhoons in the Bohai Sea, and the results were very promising. When predicting SST during Typhoon In-fa, the present spatiotemporal neural network model had a mean absolute error of only 0.27 °C for a 1-day forecast time, and a correlation coefficient of 0.74. On the other hand, comparison with baseline spatiotemporal neural network models shows that the present model is more effective in extracting input information. The current model has resolved the constraint that its predictive capability does not diminish as the forecast time increases, thereby enhancing the rationality of the prediction outcomes. In conclusion, the present spatiotemporal neural network model demonstrates its efficacy in forecasting typhoon-induced SST in the Bohai Sea, offering valuable insights for applications in meteorology and oceanography.

Long-term drought prediction using deep neural networks based on geospatial weather data

The problem of high-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance. Yet, it is still unsolved with reasonable accuracy due to data complexity and aridity stochasticity. We tackle drought data by introducing an end-to-end approach that adopts a spatio-temporal neural network model with accessible open monthly climate data as the input. Our systematic research employs diverse proposed models and five distinct environmental regions as a testbed to evaluate the efficacy of the Palmer Drought Severity Index (PDSI) prediction. Key aggregated findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts. At the same time, the Convolutional LSTM excels in longer-term forecasting. Both models achieved high ROC AUC scores: 0.948 for one month ahead and 0.617 for twelve months ahead forecasts, becoming closer to perfect ROC-AUC by 54% and 16%, respectively, c.t. classic approaches.

Workload prediction based on improved error correlation logistic regression algorithm and Cross‐TRCN of spatiotemporal neural network

AbstractIn view of the randomness of user network usage behavior in data centers, which leads to a large randomness in power load, and considering that a single randomness processing method is usually difficult to fully characterize the uncertain characteristics of the system, this paper proposes a dual fusion prediction analysis model based on an improved error correlation logic regression algorithm and a novel spatiotemporal neural network structure called Cross‐TRCN. Two weight coefficients λ1 and λ2 are introduced to fuse the prediction results with different long‐term sequence prediction performance, thereby further eliminating the influence of random errors. The results show that it is feasible to predict the workload of data centers based on the improved error correlation logic regression algorithm and the innovative spatiotemporal neural network structure Cross‐TRCN.

Physics-guided spatio–temporal neural network for predicting dissolved oxygen concentration in rivers

The prediction of river water quality is key in water resource management. Data-driven machine learning models have been widely used for predicting river water quality. However, these models seldom consider the physical mechanisms of water quality variation, which degrades the accuracy and stability of the prediction results. Hence, we develop a physics-guided spatio–temporal neural network (PGSTNN) model to predict a critical parameter for water quality assessment, i.e. dissolved oxygen. Physical information regarding spatio–temporal interactions in a hydrological network is explicitly considered to construct the architecture of PGSTNN. Two physical rules of dissolved oxygen variation (i.e. Henry’s law and power-scaling law) are established for the loss function of PGSTNN to guarantee the physical consistency of the prediction results. Experiments on the 2020–2021 water quality dataset in Atlanta, USA show that PGSTNN outperforms seven baseline neural network models in terms of prediction accuracy and stability. PGSTNN typically brings at least 10% accuracy (e.g. root mean square error and mean absolute error) improvement over the comparison methods. The proposed PGSTNN may not only improve the emergency response ability of water resource management, but also provide useful ideas for integrating scientific knowledge with machine learning.

Enhancing traffic flow prediction: Dual-branch graph convolutional network with rough data inference and adaptive spatial dependencies

Traffic flow prediction is a significant application of deep learning in spatio-temporal forecasting analysis. Existing research faces challenges that hinder prediction accuracy. One challenge is the inadequate capturing of spatial dependencies in traffic flow due to fixed pre-defined graph structures. Moreover, manually designed prior graphs still have limitations in extracting spatial features. Another challenge is the instability in short-term prediction performance when pre-defined graphs are completely abandoned in favor of parameter training. Additionally, ordinary RNN sequence convolution methods also struggle to capture long-term sequential patterns in large historical traffic data, leading to gradient vanishing or exploding issues. To address these challenges, we proposes a graph convolutional network for traffic flow prediction. We combine an improved prior graph with an adaptive graph to form a dual-branch spatio-temporal neural network. In the first branch, we introduce a time graph based on rough data inference to complement the predefined static graph. In the second branch, we construct an adaptive learning framework that dynamically learns the adjacency matrix and captures global road information. By utilizing enhanced multi-scale gated convolution, we extract spatio-temporal dependencies. Our method surpasses most baseline approaches according to extensive experiments conducted on public datasets.

A probabilistic spatio-temporal neural network to forecast COVID-19 counts

AbstractGeo-referenced and temporal data are becoming more and more ubiquitous in a wide range of fields such as medicine and economics. Particularly in the realm of medical research, spatio-temporal data play a pivotal role in tracking and understanding the spread and dynamics of diseases, enabling researchers to predict outbreaks, identify hot spots, and formulate effective intervention strategies. To forecast these types of data we propose a Probabilistic Spatio-Temporal Neural Network that (1) estimates, with computational efficiency, models with spatial and temporal components; and (2) combines the flexibility of a Neural Network—which is free from distributional assumptions—with the uncertainty quantification of probabilistic models. Our architecture is compared with the established INLA method, as well as with other baseline models, on COVID-19 data from Italian regions. Our empirical analysis demonstrates the superior predictive effectiveness of our method across multiple temporal ranges and offers insights for shaping targeted health interventions and strategies.

LSTTN: A Long-Short Term Transformer-based spatiotemporal neural network for traffic flow forecasting

Accurate traffic forecasting is a fundamental problem in intelligent transportation systems and learning long-range traffic representations with key information through spatiotemporal graph neural networks (STGNNs) is a basic assumption of current traffic flow prediction models. However, due to structural limitations, existing STGNNs can only utilize short-range traffic flow data; therefore, the models cannot adequately learn the complex trends and periodic features in traffic flow. Besides, it is challenging to extract the key temporal information from the long historical traffic series and obtain a compact representation. To solve the above problems, we propose a novel LSTTN (Long-Short Term Transformer-based Network) framework comprehensively considering the long- and short-term features in historical traffic flow. First, we employ a masked subseries Transformer to infer the content of masked subseries from a small portion of unmasked subseries and their temporal context in a pretraining manner, forcing the model to efficiently learn compressed and contextual subseries temporal representations from long historical series. Then, based on the learned representations, long-term trend is extracted by using stacked 1D dilated convolution layers, and periodic features are extracted by dynamic graph convolution layers. For the difficulties in making time-step level prediction, LSTTN adopts a short-term trend extractor to learn fine-grained short-term temporal features. Finally, LSTTN fuses the long-term trend, periodic features and short-term features to obtain the prediction results. Experiments on four real-world datasets show that in 60-minute-ahead long-term forecasting, the LSTTN model achieves a minimum improvement of 5.63% and a maximum improvement of 16.78% over baseline models. The source code is availble at https://github.com/GeoX-Lab/LSTTN.