Deep reinforcement learning based model-free optimization for unit commitment against wind power uncertainty

Abstract

Solving the unit commitment (UC) problem in a computationally efficient manner has become increasingly crucial, especially in the context of high renewable energy penetration. This paper tackles this challenge by employing the offline training of a model-free deep reinforcement learning (DRL) framework, thereby enhancing the optimization efficiency of the UC problem. The complex modeling of random variables is avoided by reformulating the UC problem as a Markov decision process, where the DRL-based method extracts knowledge regarding wind output forecasting errors from historical data. Finally, a discrete proximal policy optimization (PPO-D) algorithm is developed to generate UC solutions under the discrete action spaces necessitated by unit start-up/shut-down variables. Simulation results on the 5-unit system demonstrate that the proposed DRL-based UC model can yield an optimal solution with higher computational efficiency compared to the conventional mathematical optimization methods, while hedging against the wind power uncertainty. In addition, the case study on the IEEE 118-bus system involving 31 testing days further validates the generalization ability of the proposed DRL-based UC model.