Multidimensional dynamic attention for multivariate time series forecasting

Abstract

Attention-based models have been very effective in identifying important lagged variables for multivariate time series (MTS) forecasting applications. However, current attention-based models only provide static weights and do not consider the dynamic nature of predictions for multistep predictions of heterogeneous MTS. To address these limitations, this paper proposes a novel multidimensional dynamic attention (MDA) model for computing lagged variable importance. It incorporates a dynamic representation learner unit and considers multiple attention calculations to account for prediction dynamics, temporal information, and variable relations. Extensive experiments with both synthetic and real-world data demonstrate the effectiveness of the MDA model. It outperforms existing methods in the literature for sequence-to-sequence prediction of heterogeneous MTS in most cases and accurately identifies important features. MDA demonstrates enhancements up to 33% with real-world datasets. These findings demonstrate that the MDA model is a promising approach to MTS forecasting. The proposed attention mechanism can be utilised for other tasks related to MTS analysis beyond just forecasting, potentially enhancing the performance and interpretability of various MTS applications. The code is publicly available on GitHub, promoting widespread adoption and further research in this field [ Note: The GitHub link will be shared based on the paper’s acceptance].