Wind power bidding coordinated with energy storage system operation in real-time electricity market: A maximum entropy deep reinforcement learning approach

Abstract

The wind power forecasting error restricts the benefit of the wind farm in the electricity market. Considering the cooperation of wind power bidding and energy storage system (ESS) operation with uncertainty, this paper proposes a coordinated bidding/operation model for the wind farm to improve its benefits in the electricity market. The maximum entropy based deep reinforcement learning (RL) algorithm, Soft Actor-Critic (SAC) is used to construct the model. The maximum entropy framework enables the designed agent to explore various optimal possibilities, which means the learned coordinated bidding/operation strategy is more stable considering the forecasting error. Particularly, penalty terms are introduced into the benefit function to relax the constraints and improve the convergency. The case study illustrates that the learned policy can effectively improve the wind farm benefit while ensuring robustness.