An applied deep reinforcement learning approach to control active networked microgrids in smart cities with multi-level participation of battery energy storage system and electric vehicles

Abstract

This study proposed an intelligent energy management strategy for islanded networked microgrids (NMGs) in smart cities considering the renewable energy sources uncertainties and power fluctuations. Energy management of active power and frequency control approach is based on the intelligent probabilistic wavelet fuzzy neural network-deep reinforcement learning algorithm (IPWFNN-DRLA). The control strategy is formulated with deep reinforcement learning approach based on the Markov decision process and solved by the soft actor-critic algorithm. The NMG local controller (NMGLC) provides information such as the frequency, active power, power generation data, and status of the electric vehicle's battery energy storage system to the NMG central controller (NMGCC). Then the NMGCC calculates active power and frequency support based on the IPWFNN-DRLA approach and sends the results to the NMGLC. The proposed model is developed in a continuous problem-solving space with two structures of offline training and decentralized distributed operation. For this purpose, each NMG has a control agent (NMGCA) based on the IPWFNN algorithm, and the NMGCA learning model is formulated based the online back-propagation learning algorithm. The proposed approach demonstrates a computation accuracy exceeding 98%, along with a 7.82% reduction in computational burden and a 61.1% reduction in computation time compared to alternative methods.