A deep model for short-term load forecasting applying a stacked autoencoder based on LSTM supported by a multi-stage attention mechanism

Abstract

This paper presents an innovative univariate Deep LSTM-based Stacked Autoencoder (DLSTM-SAE) model for short-term load forecasting, equipped with a Multi-Stage Attention Mechanism (MSAM), including an input AM and several temporal AM in the pre-training phase. The input AM is used to capture the high-impact load sequence time steps of univariate input data. It should be noted that the model's performance is improved by increasing the network depth; however, finding the optimal network parameters is a challenging task due to the random assignment of the initial weights of the network. An unsupervised greedy layer-wise pre-training structure equipped with the MSAM is expanded to solve setting the random initial weight problem of the DLSTM-SAE model. The multi-stage temporal AM in the pre-training structure leads the DLSTM-SAE to properly learn the time dependencies related to remarkably long sequence input data and capture the temporal merit features lied in the LSTM memory. The performance of the proposed model is evaluated through various comparative tests with current prevalent models using actual energy market data New England ISO using three criteria indexes. The results show the superiority of the proposed model and its robustness in offline and online load forecasting.