Deep Q-Network Based Energy Scheduling in Retail Energy Market

Abstract

As a significant component of electric energy trades, retail electric market (REM) can effectively alleviate the pressure of load demand from the power grid. However, the load demand uncertainty of customers becomes a nodus because retail electricity providers (REPs) should predict the load demand when trading with wholesaler electricity provider (WEP) based on the interaction. Therefore, in this paper, we propose an optimal energy scheduling scheme in REM with consideration of the influence of decisions made in pre-purchase stages to situations in real-time stages. Firstly, we present a trading framework to analyze the strategies of REPs in REM, in which REPs conduct both pre-purchase trading with WEP, and real-time trading with customers. Then, to solve the scheduling problem caused by the demand uncertainty of customers, we design a power allocation mechanism based on the charging demand degree, by which REPs can minimize the operating cost while ensuring that each electric vehicle can be charged with sufficient energy. Next, to minimize the cost of REPs in the pre-purchase stage, we adopt deep Q-network (DQN) algorithm to implement the pre-purchase schedule. The charging station adjusts the pre-purchased schedule for each period through Q-learning and utilizes the optimal strategy to design the electricity schedule. Finally, simulation experiments show that the proposal can obtain the optimal strategy to significantly reduce the operating costs of REP.