Exploring LSTM-based Attention Mechanisms with PSO and Grid Search under Different Normalization Techniques for Energy demands Time Series Forecasting

Abstract

Advanced analytical approaches are required to accurately forecast the energy sector's rising complexity and volume of time series data. This research aims to forecast the energy demand utilizing sophisticated Long Short-Term Memory (LSTM) configurations with Attention mechanisms (Att), Grid search, and Particle Swarm Optimization (PSO). In addition, the study also examines the influence of Min-Max and Z-Score normalization approaches in the preprocessing stage on the accuracy performances of the baselines and the proposed models. PSO and Grid Search techniques are used to select the best hyperparameters for LSTM models, while the attention mechanism selects the important input for the LSTM. The research compares the performance of baselines (LSTM, Grid-search-LSTM, and PSO-LSTM) and proposes models (Att-LSTM, Att-Grid-search-LSTM, and Att-PSO-LSTM) based on MAPE, RMSE, and R2 metrics into two scenarios normalization: Min-Max, and Z-Score. The results show that all models with Min-Max normalization have better MAPE, RMSE, and R2 than those with Z-Score. The best model performance is shown in Att-PSO-LSTM MAPE 3.1135, RMSE 0.0551, and R2 0.9233, followed by Att-Grid-search-LSTM, Att-LSTM, PSO-LSTM, Grid-search-LSTM, and LSTM. These findings emphasize the effectiveness of attention mechanisms in improving model predictions and the influence of normalization methods on model performance. This study's novel approach provides valuable insights into time series forecasting in energy demands.