Self-Attentive Moving Average for Time Series Prediction

Abstract

Time series prediction has been studied for decades due to its potential in a wide range of applications. As one of the most popular technical indicators, moving average summarizes the overall changing patterns over a past period and is frequently used to predict the future trend of time series. However, traditional moving average indicators are calculated by averaging the time series data with equal or predefined weights, and ignore the subtle difference in the importance of different time steps. Moreover, unchanged data weights will be applied across different time series, regardless of the differences in their inherent characteristics. In addition, the interaction between different dimensions of different indicators is ignored when using the moving averages of different scales to predict future trends. In this paper, we propose a learning-based moving average indicator, called the self-attentive moving average (SAMA). After encoding the input signals of time series based on recurrent neural networks, we introduce the self-attention mechanism to adaptively determine the data weights at different time steps for calculating the moving average. Furthermore, we use multiple self-attention heads to model the SAMA indicators of different scales, and finally combine them through a bilinear fusion network for time series prediction. Extensive experiments on two real-world datasets demonstrate the effectiveness of our approach. The data and codes of our work have been released.