Deep Reinforcement Learning Based Bidding Strategy for EVAs in Local Energy Market Considering Information Asymmetry

Abstract

With the increasing penetration of distributed energy resources (DERs) in smart grids, customers can be aggregated to participate in the local energy market (LEM). In the LEM, on the one hand, the aggregated customers can purchase electricity from local DERs at a price that may be lower than the electricity price from the utility. On the other hand, when there is abundant energy, the aggregated customers can sell them in the LEM at a higher price, supplementing the grid power supply with clean renewable energy. Therefore, the customers' dependency on the utility is reduced. In this context, this article presents a bidding strategy for electric vehicle aggregators (EVAs) based on data analytics and deep reinforcement learning (DRL). To achieve this goal, an asynchronous learning framework is put forward to help EVAs formulate bids, including bidding price and bidding volume. Compared with the conventional model-based strategy, the learning-based strategy shows advantages in rapid decision-making and reduced reliance on stochastic models. Besides, the EVAs can cope with the information asymmetry in the LEM by using the DRL method. A modified deep deterministic policy gradient methodology is utilized to speed up the online training to avoid high losses at the training stage. According to the simulation results, it can be concluded that the profit of the learning-based strategy is 63.3% higher than that of the random strategy. The coefficient of variation of the learning-based strategy is 76.4% lower than that of the random strategy. Therefore, the proposed learning-based method is effective.