Local demand management of charging stations using vehicle-to-vehicle service: A welfare maximization-based soft actor-critic model

Abstract

Transportation electrification has the potential to reduce carbon emissions from the transport sector. However, the increased penetration of electric vehicles (EVs) can potentially overload the distribution systems. This becomes prominent in locations with multiple EV chargers and charging stations with many EVs. Therefore, this study proposes a welfare maximization-based soft actor critic (SAC) model to mitigate transformer overload in distribution systems due to the high penetration of EVs. The demand of each charging station is managed locally to avoid network overload during peak load hours in two steps. First, a welfare maximization-based optimization model is developed to maximize the welfare of electric vehicle owners by performing vehicle-to-vehicle(V2V) service. In this step, the sensitivity of EV owners to different parameters (energy level, battery degradation, and incentives provided by fleet operators) is considered. Then, a deep reinforcement learning-based method (soft-actor critic) is trained by incorporating the welfare value (obtained from the welfare maximization model) in the reward function. The total power demand (at the transformer level) and transformer capacity are also included in the reward function. The agent (fleet operator) learns the optimal pricing strategy for local demand management of EVs by interacting with the environment. Each electric vehicle responds to the action (price) by deciding the amount of power they are willing to charge/discharge (V2V) during that interval. Training is performed offline, and the trained model can be used for real-time demand management of different types of charging stations. The simulation results have shown that the proposed method can successfully manage the demand of different charging stations, via V2V, without violating the transformer capacity limits.