Multi-step forecasting of multivariate time series using multi-attention collaborative network

Abstract

Multi-step forecasting of multivariate time series plays a critical role in many fields, such as disaster warning and financial analysis. While attention-based recurrent neural networks (RNNs) achieved encouraging performance, two limitations exist in current models: i) Existing approaches merely focus on variables’ interactions, and ignore the negative noise of non-predictive variables, ii) These methods cannot model the difference in the temporal importance of the target and the non-predictive series to prediction. To tackle these challenges, we propose a triangle structured Multi-Attention Collaborative Network (MACN), which includes a backbone network T-net with attention-based encoder–decoder framework, and an auxiliary hierarchical network NP-net. NP-net focuses on non-predictive variables, capturing the most relevant variables and temporal dependencies through the proposed variables-distillation attention network (VDN) and long short-term memory network (LSTM). T-net executes on target variable, and its encoder and decoder are both connected to NP-net, thereby using the output of NP-net to assist learning and decision-making. Specifically we design a knowledge-enhanced LSTM (KeLSTM) as the encoder and decoder of T-net. In the coding stage, KeLSTM refines the output of NP-net to strengthen the latent semantics of the target variable. In the decoding stage, KeLSTM captures subtle differences between the target and the non-predictive variables’ contribution to prediction, and improves model’s predictive ability by alleviating such conflicts. Experiments on three real-world datasets demonstrate that MACN outperforms different types of state-of-the-art methods.