Abstract
With the rapid development of information and communications technology and high penetration of renewable energy, the role of an aggregator in a smart grid has emerged to better coordinate power and cash flows between energy producers and consumers through the adjustment of pricing signals. This study proposes variation indices about the statistics of renewables and a control law for an energy storage system. A deep reinforcement learning based pricing strategy of an aggregator for profit maximization in consideration of the energy balance is developed accordingly. The proposed approach can consider opponents’ behaviors, variability of renewables, and varying bounds of charging and discharging events in a nonstationary environment, which can be hardly addressed by pricing strategies based on conventional learning algorithms such as Q-learning and deep Q-network. Numerical analysis using real-world data shows that the proposed approach can outperform existing pricing strategies in terms of the learning speed and profit of aggregators.
Highlights
O WING to the advance of information and communications technologies and various types of power generation, smart grids have turned into a complex distributed network [1]
To illustrate the effectiveness of our control law, deep deterministic policy gradient (DDPG) without energy storage system (ESS) control was involved for comparisons; in that case, the dimension of the action space was increased by one to include the charging and discharging actions, and simple truncation was used to meet the physical bounds of the ESS
This paper investigated the pricing strategy of an aggregator for profit maximization under uncertainty caused by hybrid energy sources, customer behaviors, and pricing strategies of opponent aggregators
Summary
O WING to the advance of information and communications technologies and various types of power generation, smart grids have turned into a complex distributed network [1]. Integrating the use of renewable energy and an energy storage system (ESS) into the formulation of competitive pricing strategies for an aggregator within the deep reinforcement learning framework becomes increasingly important for at least two reasons. Using an ESS as a buffer can provide an aggregator with flexibility for energy management It introduces additional storage system dynamics involving charging and discharging actions that have varying bounds; these varying bounds cannot be directly addressed by deep learning methods considering continuous action space or price space. Motivated by the idea of DDPG and considering the integration of renewable energy and an ESS, we propose a deep reinforcement learning based approach for a pricing strategy of an aggregator in a competitive energy market. We propose a deep reinforcement learning based pricing strategy of an aggregator, which uses the variation indices and control law for the ESS.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IEEE Transactions on Emerging Topics in Computational Intelligence
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
