Intelligent EV Charging for Urban Prosumer Communities: An Auction and Multi-Agent Deep Reinforcement Learning Approach

Abstract

Recently, the deployment of electric vehicles supply equipment (EVSE) and its market is expanding rapidly to support the massive penetration of electric vehicles (EVs). However, to accomplish an effective EV charging mechanism for urban prosumer communities, it is imperative to tackle the challenges of distinct energy generation among the communities, dependency of the total purchasable energy price of each EV based on the distance between EV and EVSE, and extreme uncertainty among the energy demand and generation. Therefore, in this paper, the problem of EV charging of urban prosumer communities is studied. In particular, a joint optimization problem is proposed to maximize both the social welfare and EV charging achieved rate of the considered urban prosumer communities. Consequently, the formulated problem is decomposed into 1) truthful double auction problem for determining the unit price and winners by maximizing social welfare, and 2) EV auction losers charging problem for improving EVs charging achieved rate by purchasing energy from the power grid. Then the breakeven-based double auction (BDA) mechanism is proposed to find the unit price and EV winners’ for charging. Sequentially, a multi-agent deep reinforcement learning-based asynchronous advantage actor-critic algorithm with a long short-term memory layer (A3C-LSTM) is adopted to achieve the optimal grid energy buying decision for ensuring the charging of the losers. Finally, the experimental results demonstrate the efficacy of the proposed model that can increase the number of EV charging up to 57.31%, and prosumer communities have gained 86.04% of their income compared to baseline methods.