Deep reinforcement learning for energy trading and load scheduling in residential peer-to-peer energy trading market

Abstract

The popularization of solar generation enables residential households to supply their loads and trade the surplus energy through residential peer-to-peer (P2P) energy trading market. In a residential community with multi-households, households are able to determine demand-side management (DSM) and energy trading strategies to reduce electricity cost. However, residential users often decide market participation biddings according to their experience and guesses on other households’ biddings, and it is difficult to know how far these biddings are from the best possible market biddings achievable by the P2P market trading mechanism. Targeting the maximum cost savings achievable from this P2P market, this paper focuses on how to obtain such a maximum saving under optimal energy trading and DSM strategies using a multi-agent deep reinforcement learning approach to obtain the optimal solution. In this regard, the energy trading and DSM problem for residential households is formulated as a partially observable Markov decision process (POMDP). Then, a reinforcement learning algorithm with decentralized training and execution approach is adopted in the proposed POMDP problem to fit the policy function and value function, which are trained against each other to learn the optimal strategies by trial-and-error. The simulation result validates that by using the proposed multi-agent reinforcement learning method to optimize the P2P energy trading and DSM strategies, the average reward for households can be significantly improved by 8.52% and 14.03%, respectively, compared with other two state-of-the-art deep reinforcement learning methods.