Hyperparameter optimization strategies for machine learning-based stochastic energy efficient scheduling in cyber-physical production systems

Abstract

One of the major challenges faced by system operators in an integrated energy system (IES) is to come up with an optimal schedule of the energy generation units in the presence of uncertainties at both the demand and supply sides. Employing efficient forecasts of the uncertain parameters while ensuring the least possible prediction error is crucial to the overall scheduling of IESs. Long Short-Term Memory (LSTM), a Recurrent Neural Network (RNN) based architecture is widely used in natural language processing and time series forecasting and has reported to be a viable and powerful forecasting technique. The aim of this work is to enhance the autonomy of energy scheduling in cyber-physical production systems (CPPS) through the development of data-driven LSTM based forecast models for the uncertain parameters. However, one of the demanding tasks in time series forecasting methods such as LSTM and the subject that received little attention is the judicious choice/fine-tuning of the hyperparameter values, which significantly impacts the performance of the developed forecasting models. This paper mainly focuses on the performance analysis of various hyperparameter tuning techniques and algorithms used by LSTM networks in forecasting uncertain parameters for stochastic energy scheduling in the context of smart manufacturing. The plant under consideration is the highly energy intensive Model Factory (MF) at the Singapore Institute of Manufacturing Technology (SIMTech), that features an actual production environment supporting both experimentation and learning of digitalisation technologies for Industry 4.0. Three main energy sources installed in an IES framework consisting of solar photo voltaic (PV) system, waste-to-energy (WTE) unit and the main utility grid serves as the energy supply module to meet the requested energy by the industrial facility. Various strategies to tune the LSTM hyperparameters in forecasting the uncertain parameters are examined. Among the four strategies, the first one deals with the traditional manual tuning approach and the second one focuses on automating the traditional manual approach using a FOR loop. While the third one focuses on a tuning strategy using Optuna with a grid search algorithm, the fourth strategy concentrates on adopting Optuna with a Bayesian optimization framework. The overall objective entails generating an optimal day-ahead forecast of the uncertain parameters (in this case solar PV irradiance, temperature, main utility grid energy price and the energy demand of the MF) using LSTM based on the best performing hyperparameter tuning techniques. These forecasts are then utilized to solve a day-ahead stochastic energy scheduling problem that generates an optimal energy dispatch schedule with the overall objectives to minimize the total operational cost and carbon emission.