Abstract
Multivariate time series forecasting has long been a research hotspot because of its wide range of application scenarios. However, the dynamics and multiple patterns of spatiotemporal dependencies make this problem challenging. Most existing methods suffer from two major shortcomings: (1) They ignore the local context semantics when modeling temporal dependencies. (2) They lack the ability to capture the spatial dependencies of multiple patterns. To tackle such issues, we propose a novel Transformer-based model for multivariate time series forecasting, called the spatial–temporal convolutional Transformer network (STCTN). STCTN mainly consists of two novel attention mechanisms to respectively model temporal and spatial dependencies. Local-range convolutional attention mechanism is proposed in STCTN to simultaneously focus on both global and local context temporal dependencies at the sequence level, which addresses the first shortcoming. Group-range convolutional attention mechanism is designed to model multiple spatial dependency patterns at graph level, as well as reduce the computation and memory complexity, which addresses the second shortcoming. Continuous positional encoding is proposed to link the historical observations and predicted future values in positional encoding, which also improves the forecasting performance. Extensive experiments on six real-world datasets show that the proposed STCTN outperforms the start-of-the-art methods and is more robust to nonsmooth time series data.
Highlights
Time series forecasting has a wide range of application scenarios in transportation, finance, medical, and other fields
We propose a novel Transformer-based model for multivariate time series forecasting, called the spatial– temporal convolutional Transformer network (STCTN)
The obstacles of applying Transformer to multivariate time series forecasting are that the standard self-attention mechanism is only used at the sequence level and cannot capture the spatial dependencies, and it is weak in capturing the temporal dependencies of multiple patterns
Summary
Time series forecasting has a wide range of application scenarios in transportation, finance, medical, and other fields. The development of graph neural networks (GNNs) has brought time series forecasting to a new level and numerous GNN-based methods for spatiotemporal data prediction have been proposed, such as DCRNN [14], STGCN [15], ASTGCN [16], MTGNN [17], STSGCN [18], StemGNN [19], etc. The group-range convolutional attention mechanism uses multihead attention to learn the latent graph structures among multiple time series, extracting dynamic and multimodal spatial dependencies, which addresses the second shortcoming. We design a novel Transformer-based encoder–decoder framework for multivariate time series forecasting that can dynamically model spatiotemporal dependencies. Two novel range convolutional attention mechanisms are proposed to effectively extract dynamic and multimodal spatiotemporal dependencies and reduce the computation complexity.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.