A deep reinforcement learning approach for power management of battery-assisted fast-charging EV hubs participating in day-ahead and real-time electricity markets

Abstract

Publicly available electric vehicle charging hubs are expected to grow, to meet the increasing charging demand of EVs. A dominant class of these will be fast-charging hubs where the EVs will arrive for charging at all hours of the day, get the requested charge, and leave promptly. The profitability of these fast-charging hubs will be highly dependent on the variation of the day-ahead prices of electricity, volatility of the real-time power market, and the randomness of EV charging demand. The hubs can hedge against these uncertainties by committing power purchases in the day-ahead electricity market and by adopting dynamic real-time power management strategies. We develop a novel two-step methodology. The first step entails a mixed integer linear program (MILP) that assists the hubs in their day-ahead power commitment. The second step employs a Markov decision process (MDP) model that derives the real-time power management control actions. The MILP is solved using a commercial solver and the MDP is solved using a deep reinforcement learning algorithm. We demonstrate the effectiveness of our methodology for a fast-charging hub, housing 150 charging stations and a battery storage system, that operates in the Pennsylvania-New Jersey- Maryland interconnection (PJM) power grid.