Deep Reinforcement Learning for Charging Electric Vehicles With Uncertain Photovoltaic Power

Abstract

Electric vehicles (EVs) are some of the major consumers of renewable energy generation in recent years. However, uncertainties associated with the photovoltaic (PV) output and EV users’ charging behaviors have imposed great challenges on PV power consumption. This study presents a real-time EV charging strategy based on deep reinforcement learning (DRL) to reduce the PV curtailment and charging costs of EV users. A mathematical model is constructed to describe the charging control process of EVs, in which the uncertainties of PV outputs and users’ charging behaviors are considered. This EV charging control problem is then formulated as a Markov Decision Process, and DRL is developed to learn the optimal real-time charging strategy in a dynamic environment. A policy search algorithm based on Proximal Policy Optimization is used to train the neural network. Moreover, a novel allocation criterion is proposed for the charging behavior of a single EV, thus ensuring that each EV is fully charged before departure. The proposed approach is tested on simulation cases, and the results verify the effectiveness in improving PV accommodation with uncertainties.