Power Flow Management in Electric Vehicles Charging Station Using Reinforcement Learning

Abstract

This paper investigates optimal power flow management problem in an electric vehicle charging station. The charging station is powered by solar PV and is tied to the grid and a battery storage system through necessary power conversion interfaces for DC fast charging. The optimal power management problem for EV charging is solved via reinforcement learning (RL). Unlike classical optimization methods such as dynamic programming, linear programming (LP) and mixed-integer linear programming which are limited in handling stochastic problems adequately and are slow due to the curse of dimensionality when used for large dynamic problems, RL does not have to iterate for every time step as learning can be done completely offline and optimal solutions saved in a lookup table, from which optimal control actions can be retrieved almost instantaneously. The optimization problem in this paper is defined as a Markov Decision Process (MDP) and a modified Q-learning algorithm that indexes both states and control actions in a hash-table (dictionary) fashion is used to solve it. The algorithm is tested with a typical load curve over a 24-hour horizon. The simulations results demonstrate that the modified Q-learning algorithm achieves higher total rewards and returns a 14% lower global cost than the conventional Q-learning formulation.