A Multiagent Deep Reinforcement Learning Based Approach for the Optimization of Transformer Life Using Coordinated Electric Vehicles

Abstract

The uncertainties of charging behavior of electric vehicle (EV) owners have a negative impact on the loss of life (LOL) of distribution transformer. This article proposes a decentralized EV charging framework for optimization of the LOL of distribution transformer considering the dissatisfactions of EV owners. Specifically, long-short-term memory (LSTM) neural network is first utilized to capture the uncertainties caused by the load demand and electricity price. After that, each EV is modeled as an intelligent agent and a multiagent deep reinforcement learning approach is applied to solve the coordinated charging problem based on the forecasting information by the LSTM network. All the agents are trained in a centralized manner to develop coordinated control strategies while informing decisions based on local information when finishing the training process. The proposed approach can achieve coordinated charging management of EVs based on local information, which helps preserve the privacy of EV owners, reduce the cost induced by the deployment of communication devices, and avoid single-point failure. In addition, the parameter space noise and deep dense architecture in reinforcement learning are introduced to overcome premature convergence, training instability, and inefficiency due to the large action space of multiagent scenario. Comparative tests are carried out among several benchmarks utilizing real-world data to illustrate the effectiveness of the proposed approach.