A Graph Reinforcement Learning-Based Decision-Making Platform for Real-Time Charging Navigation of Urban Electric Vehicles

Abstract

To provide efficient charging behavior decision-making for urban electric vehicles (EVs), this article proposes a new platform for real-time EV charging navigation (EVCN) based on graph reinforcement learning. Considering the interaction of EVs with charging stations (CSs) and traffic networks, the navigation goal of the “vehicle-station-network” coupled system is to minimize the charging cost and traveling time of EV owners. Specifically, to realize data acquisition and decision-making output, we first characterize the EV charging and traveling behavior as the dynamic interaction process of graph-structured networks. A graph convolutional network is used to extract the environment information required for EVCN, and the generated environment feature is fed into the underlying network of deep reinforcement learning (DRL), which can help the agent better understand massive graph-structured data. Then, the real-time navigation problem is duly formulated as a finite Markov decision process. A sequential scheduling pattern is built according to the sorting of EV charging urgency and solved by a Rainbow-based DRL algorithm. It achieves the sequential recommendation of CSs and planning of traveling routes for multiple EVs. Case studies are conducted within a practical zone in Nanjing, China. Simulation results verify the developed platform and the solving method.