Attention-based gating optimization network for multivariate time series prediction

Abstract

Multivariate time series prediction is helpful for scientific decision-making and reliable assessments in numerous fields. Capturing time series nonlinear change rules with increasing dependency complexity among multivariate time series remains challenging. We propose an attention-based gating optimization network (AGON) to solve this issue. To relieve the conflict between external and target factors, the AGON divides the original time series into external factors and target factor for two parallel encoder inputs. One encoder employs feature-perceived attention to distinguish external factors’ contributions and limit the impact of low-contributing factors on high-level semantics. The other leverages long short term memory (LSTM) to establish the temporal dependencies of the target factor. In the decoder phase, fine-grained denoising attention is designed to assign an attention weight vector instead of a single scalar to filter fine-grained harmful noise in the hidden state to the uttermost. Finally, information fusion LSTM is introduced to maintain the output balance between external and target factors. An extensive experiment of three fields of energy, environment, and finance (wind speed, PM2.5, four stock datasets) is presented. AGON is able to achieve 38%, 37%, and 46% improvements in the average MAE, RMSE, and MAPE results over state-of-the-art methods across all three datasets and three-step-ahead forecasting, which implies that AGON can be very competitive and holds versatility for diverse industries’ multivariate time series even if the dataset presents strong fluctuation and a nonlinear changing tendency.