Temporal attention convolution network for energy time series forecasting

Abstract

Energy time series are characterized by stochasticity, short-term periodicity, and long-term periodicity. Many traditional models such as the autoregressive model, Gaussian process regression are unable to capture the complex time relationships of energy time series. Recurrent neural networks (RNNs), which are widely used for energy time series forecasting, are prone to gradient disappearance or gradient explosion when processing a long sequence data. Thus, RNNs are failed to capture the complete information of the whole long sequence data. To extract complex irregular patterns of energy time series and selectively learn spatio-temporal features, a Temporal Attention Convolution Network (TACN) is proposed in this paper. This structure can extract spatial and temporal features of energy time series, which full filled with our demand. Firstly, to solve the problem that RNNs are prone to information loss when processing the long sequence data, a temporal convolutional network (TCN) is introduced. TCN enables the output to capture the complete information of the whole long sequence data. Then, to reinforce the TCN for extracting effective spatio-temporal features, Squeeze-and-Excitation Network (SENet), a module for computing channel attention, was introduced to this TCN structure. By applying this module, the ability of the acquisition of important channel information has been enhanced. Finally, to further improve the robustness of the model, an autoregressive linear module is constructed to capture the linear relationship among the variables. Experiments on the publicly available datasets Electricity and Solar-Energy show that compare with existing deep learning models, TACN achieves more ideal performance.