Deep Reinforcement Learning-Based Spatiotemporal Decision of Utility-Scale Highway Portable Energy Storage Systems

Abstract

Mobile charging is an efficient solution to meet peak charging demand on highways. In this paper we propose a deep reinforcement learning (DRL)-based approach to maximize the revenue of a utility-scale highway portable energy storage system (PESS) for on-demand electric vehicle charging. We consider a PESS that consists of an electric truck, battery, and charging stations on highways. Actions include the selection of stations and routes, the selection of charging and discharging, and the corresponding charging/discharging power selection. The first two are discrete, while the last is continuous. To deal with the hybrid action space and time-varying state space of PESS, we design an action space, observation space, and reward function according to its characteristics; in addition, we develop a state-of-the-art DRL model for online decision-making considering the uncertainty in the real-time market price of electricity. Through numerical simulations with real-world electricity price data in California, we demonstrate the effectiveness of the proposed method.